Talk:Progress in artificial intelligence

Excuse me, but why are all of these board games listed? There is no explanation for what they are supposed to represent. It is useless information for the article. Contralya (talk) 17:33, 28 April 2008 (UTC)

The article shows problems that have been solved to varying degrees by artificial intelligence researchers and indicates the level of performance one can achieve with an AI algorithm. Board games are well-defined problems and so are easier to tackle than more open-ended problems (generally speaking). While the results might appear useless to some they have required advances in the theory and practice of AI and are important in that respect. You have to walk before you can run. Pgr94 (talk) 17:46, 28 April 2008 (UTC)

Machine translation

How should machine translation be classified? People knowing a second language often perform better than machine translation systems but the average person might be less capable. Also, if we take into account multiple languages, machine translation systems like google translate would significantly outperform most humans. How many people can translate from arabic -> polish and french -> swahili and armenian -> vietnamese. Over several languages machines win hands-down. pgr94 (talk) 22:22, 26 May 2010 (UTC)

I think it should stay where it is might it could use a comment noting machines higher versitality in the field as you noted? —Preceding unsigned comment added by 83.250.36.199 (talk) 19:09, 1 August 2010 (UTC)

I added a comment to the article as suggested. Feel free to improve on it. pgr94 (talk) 22:27, 1 August 2010 (UTC)

Rubics Cube

There has been some back and fourth editing regarding rubics cube and there seems to be a disperancy betwen users regarding what solving rubics cube constitutes to.

Since the optimal solution, aka the one requireing the least ammount of moves from every starting position is not yet known, i would list the game as strong super human but not optimal, as currently is. —Preceding unsigned comment added by 83.250.36.199 (talk) 19:02, 1 August 2010 (UTC)

In Optimal solutions for Rubik's Cube it says: "In 1997 Richard Korf[5] announced an algorithm with which he had optimally solved random instances of the cube." Are you suggesting that this is wrong or that it is optimal only for some cube starting positions? pgr94 (talk) 22:15, 1 August 2010 (UTC)

Classification

In an edit comment Charles Gillingham asked who developed the 5-level classification:

optimal: it is not possible to perform better

strong super-human: performs better than all humans

super-human: performs better than most humans

par-human: performs similarly to most humans

sub-human: performs worse than most humans

The short answer is, I did. However, the levels are explicitly mentioned in the various articles cited. I suspect we won't find a single reference that mentions them all, but they are there in the literature. My edits may be running a bit close to WP:SYN, so I'm open to other opinions on the matter. pgr94 (talk) 09:57, 30 January 2011 (UTC)

I suppose you could argue that it's commonsense. It's certainly not some egregious violation of WP:SYN, in my view anyway. Thanks for being so straight forward about it. I only noticed it because I'm going through the AI article and redoing the citations and along the way I ran into a few topics that were missing a citation and this was one of them. I'll leave the tag in AI and remove it here. ---- CharlesGillingham (talk) 05:48, 31 January 2011 (UTC)

Game shows

Today Watson beat the two Jeopardy champions. It is well above merely "par-human" —Preceding unsigned comment added by 173.88.249.116 (talk) 01:03, 17 February 2011 (UTC)

Watson was also beaten by a U.S. congressman: http://www.datacenterknowledge.com/archives/2011/03/01/congressman-defeats-ibms-watson-in-jeopardy/. It also doesn't do speech recognition, so it plays with an unfair advantage against human players. Admittedly, the congressman is a former Jeopardy champion. But should we be citing a couple Watson victories to support Watson's playing at superhuman level? I figure the claim is probably right, so I'm not removing it, but it would be nice to see a source that talks explicitly about fair comparisons between Watson's play and most human play. However, here are two other bases for doubt: (1) I've heard (only by word of mouth) that Watson can be beaten by an average human who knows how to exploit a couple weaknesses in Watson's play; (2) Watson depends heavily on human prepping, so some claim it might be better to call it a supplement to human intelligence and not really artificial intelligence. Wikipedia really shouldn't be deciding this question. —Ben Kovitz (talk) 22:49, 7 August 2011 (UTC)

Go citation

The citation for computers being par-human at go is wikipedia itself. Is this acceptable? (could not find the wikipedia guideline on this)

No, such a citation is not acceptable: WP:CIRCULAR. I just removed it. The claim is clearly true, but I wasn't able to (quickly) find a source that supports it directly. Can you find one? —Ben Kovitz (talk) 22:27, 7 August 2011 (UTC)

Driving cars

At the moment driverless cars are listed both in the section "Super-human" ("Driving a car: super-human. Google driverless cars are safer and smoother when steering themselves than when a human takes the wheel.") and "Sub-human" ("Autonomous driverless cars"). One of the two has to be wrong? 91.229.57.240 (talk) 18:05, 2 December 2013 (UTC)

I'm pretty sure anyone familiar with the state of the art would agree that autonomous cars are sub-human at this point. For instance, they are not yet able to drive safely in inclement weather. And the fact is that Google's cars still have to hand over the controls to humans when they're not sure what to do. They do by design drive much more conservatively than most humans, which is enough to explain the claims in the citation. Yak314 (talk) 13:51, 11 February 2016 (UTC)

I moved Safely driving a car from sub human to super human. Waymo has driven more than a million miles since January 2018 without causing a single accident. The average driver has caused many more accidents in a million miles. --Mschribr (talk) 18:41, 26 May 2018 (UTC)

Driving a million miles without causing an accident isn't superhuman, especially if we're not counting accidents the driver didn't cause. I moved it to par-human for now and marked caution around this claim as it has not yet been definitively proven. We need a reliable source will state that a particular car is safer on average than human driving in some domain to move it to "high human"; forbes sites blogs aren't generally considered reliable sources on Wikipedia. Rolf H Nelson (talk) 01:42, 31 May 2018 (UTC)

We can include accidents the driver did not cause. Since May 2017, Waymo has driven 4 million miles. That will be going up by more than 1 million every 2 months. In that time Waymo has not caused a single accident. Waymo has been involved in 5 accidents. How many humans have driven 4 million miles? Of those how many have been in 5 or less accidents? Find 1 person. Humans cannot match the driving record of Waymo. Waymo is superhuman at driving safely. --Mschribr (talk) 16:25, 19 June 2018 (UTC)

185 crashes per 100 million VMT

Human crash rates in 2011 were 185 crashes per 100 million miles according to [1], so 5 crashes in 4 million miles isn't even far from statistical error of that range. Driverless cars get a lot of coverage; I'm sure once evidence shows that driverless cars are safer than human drivers in some specific domain the media will cover that. Rolf H Nelson (talk) 18:41, 14 July 2018 (UTC)

The 5 crashes involving Waymo are not in the same class as the human crashes. Humans caused the 5 Waymo crashes. Wayno drove 4 million miles and caused zero crashes. In 2011 humans drove 100 million miles and humans caused 185 crashes. So Waymo is not in the statistical error range of any human driver. Waymo is much better than any human driver. The evidence is here. Waymo driving is better than humans driving. The media prefers accidents over safe driving, so Waymo may not be reported. --Mschribr (talk) 07:15, 15 July 2018 (UTC)

No, the media does report on overall safety (for example, [2] in May), it's just that their analysis has so far differed from yours. Rolf H Nelson (talk) 17:49, 5 August 2018 (UTC)

Waymo has driven another million miles in the last month without an accident. Since May 2017, Waymo has driven 5 million miles. Since it has not had an accident in 5 million miles I will move Waymo from par human to high human. — Preceding unsigned comment added by Mschribr (talk • contribs) 11:58, 27 July 2018 (UTC)

Maybe, but I don't know how much safety-driver intervention in those 5 million miles has prevented accidents (for example, if there are any "tries to drive into a stationary object" problems that allegedly plague Tesla). I'd ask how many miles have been driven by fully driverless Waymo cars, but again, this seems like WP:OR. Rolf H Nelson (talk) 17:49, 5 August 2018 (UTC)

I can tell you how many safety-driver interventions Waymo had in Arizona since November 2017. Zero. Because there is no human in the driver seat to do an intervention. The Wired article cannot be used as a Wikipedia source because it is unreliable. The Wired article starts, "A SELF-DRIVING WAYMO minivan being driven by its human operator crashed in Chandler, Arizona, Friday afternoon, threatening to resurrect tough questions about the safety of autonomous technology". Waymo was hit by a reckless driver who ran a red light. Waymo was the victim and was being driven by a human driver. So how can the Wired article use the crash to threaten to resurrect tough questions about the safety of autonomous technology? You said, Wired said, "some observers continue to reserve judgment whether autonomous driving safety has been adequately demonstrated". The Wired article does not say who are these observers. So we can not say these observers are authorities and able to make judgments about autonomous driving safety. Another indication that Wired is not a reliable source is the article also has mistakes in grammar that would not be made by a professional journalist. The article repeats the word the, "injuring the the SUV's female safety driver". As we saw earlier Wired makes mistakes and cannot be considered a reliable source. I will remove this counterpoint. Mschribr (talk) 15:25, 6 August 2018 (UTC)

Mschribr I don't know how many miles non-safety-driver Waymo has driven, and it's not the page editors' job to review your original research and check it for accuracy. I'm sorry we don't see eye-to-eye on whether Waymo's high-human safety has been demonstrated. Rolf H Nelson (talk) 16:27, 26 August 2018 (UTC)

Rolf H Nelson Sources tell us Waymo stopped using backup drivers in November 2017. https://www.cnet.com/roadshow/news/waymo-ditches-safety-driver-in-self-driving-public-pilot/ . Waymo has driven over 4 million miles since November 2017. https://waymo.com/ontheroad/ . Waymo has not caused any accidents since November 2017. https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/autonomousveh_ol316%2B+&cd=1&hl=en&ct=clnk&gl=us . It is the editor's job to review the articles for accuracy and original research. We do not have to see eye to eye. We need to cite credible sources. There are credible sources for my edits so it is not original research. Mschribr (talk) 14:43, 28 August 2018 (UTC)

Mschribr I asked the 3o board to provide a third opinion here (of course, other page editors can chime in too). I think the Wired opinion is admissible as a sourced opinion and that "Uber has driven xxx miles without causing an accident" is not without a source that explicitly states that. Rolf H Nelson (talk) 18:14, 1 September 2018 (UTC)

I welcome more input. Anybody concerned about the next 10 years should give their opinion. Many lives will be lost if we slow down the introduction of competent autonomous vehicles like Waymo on the road. Anybody with common sense would see the Wired article makes no sense. A reckless driver speeding through a red light looses control and hits the Waymo in the opposite direction waiting for the green light. The Wired article says this is threatening to resurrect tough questions about autonomous cars. Waymo was the victim not the cause of the accident. Rolf h nelson, you quote me as saying "Uber has driven xxx miles without causing an accident". I never said that about Uber. I said Waymo has driven over 4 million miles since November and not caused an accident. There is no human driver that could match that safety record. You said, "has driven xxx miles without causing an accident is not without a source that explicitly states that". Then you are saying there is a source that says driving x miles without causing an accident. Mschribr (talk) 04:37, 2 September 2018 (UTC)

Hi all, answering third opinion. Most of this discussion is based on personal interpretation of statistics, which is WP:SYNTH and WP:OR. I have deleted the disputed content until reliable secondary sources can be provided that characterize self-driving cars as one of the given categories. Additionally, Mschribr: ". Anybody concerned about the next 10 years should give their opinion. Many lives will be lost if we slow down the introduction of competent autonomous vehicles like Waymo on the road." This is editing with an agenda which is a clear no-no. See WP:SOAPBOX. I have zero opinion on what is the correct classification until sources are presented. ^†Basilosauridae❯❯❯Talk 02:03, 3 September 2018 (UTC)

We have a source that says safely driving a car belongs in the high-human category. See the article Data Shows Google’s Robot Cars Are Smoother, Safer Drivers Than You or I in MIT Technology Review, https://www.technologyreview.com/s/520746/data-shows-googles-robot-cars-are-smoother-safer-drivers-than-you-or-i/ Mschribr (talk) 09:33, 3 September 2018 (UTC)

the source provided is relaying a claim from a google employee and does not state that his assertion is accepted by others or that the writer even believes it. I wouldn’t consider the source provided to be sufficient to make this claim on Wikipedia. ^†Basilosauridae❯❯❯Talk 14:05, 3 September 2018 (UTC)

The employee is relaying two studies based on hundreds of thousands of miles Google vehicles drove on public roads in California and Arizona. Data showed that Google software was much better at maintaining a safe distance from the vehicle ahead than the human drivers were. Google software is spending less time in near-collision states. The source does not state that the studies are rejected by others. The writer does not say that he doubts the studies. The source is reliable and follows Wikipedia policies. Mschribr (talk) 20:46, 3 September 2018 (UTC)

I would argue that this article, for the previously stated reasons, isn't sufficient to support the claim you are making. Here are a few excerpts from the article you provided that cast doubt on the legitimacy of the claims made:

• "Tests of Google’s autonomous vehicles in California and Nevada suggests they already outperform human drivers."
• "Chris Urmson made those claims today at a robotics conference in Santa Clara, California."
• "In addition to painting a rosy picture of his vehicles’ autonomous capabilities..."
• "Although that might suggest the company is thinking about how to translate its research project into something used by real motorists, Urmson dodged a question about how that might happen."

I am waiting for response from Rolf h nelson regarding his thoughts on the discussion and source provided, but most likely we'll end up in an RfC. ^†Basilosauridae❯❯❯Talk 20:03, 4 September 2018 (UTC)

We have another source that is less controversial, that says AI safely driving a car belongs in the high-human category. See the article Virginia Tech report finds national crash rate for conventional vehicles higher than crash rate of self-driving cars in Green Car Congress, http://www.greencarcongress.com/2016/01/20160109-vtti.html Mschribr (talk) 11:55, 5 September 2018 (UTC)

The VT report doesn't appear to be peer-reviewed, correct me if I'm wrong. If it's not peer-reviewed, it would need a stronger secondary source than green car congress for inclusion, and would still need to be balanced with something like "Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?", which is peer-reviewed and has more citations. MIT Technology Review isn't the best secondary source, but I'm borderline OK with its inclusion as an opinion attributed to Google. I remain of the belief that as of September 2018, the question of whether high-human "safe driving" has been demonstrated, appears to remain disputed, and that appropriate caveats should be included, until such a time as the first-tier sources accept that self-car driving ability is above average. Rolf H Nelson (talk) 17:38, 8 September 2018 (UTC)

For now, I do not have time to pursue this thread. Mschribr (talk) 14:36, 3 October 2018 (UTC)

Most everyday tasks performed by humans

What are "Most everyday tasks performed by humans"? This is vague. Instead, can you name the everyday tasks performed by humans that an artificial intelligence does at a sub-human level? --Mschribr (talk) 14:22, 11 January 2015 (UTC)

Category names

I suggest changing the heading "Super-human" to "World class" and bringing the title "Strong super-human" back to super-human".

Having 5 classes rather than the original 4 is fine, but in most of "super-human" category, computers are not even undisputed world champions. e.g. in Bridge it's certainly world class but not the champion. If there isn't any good reason why not, I'll make the change. Chris55 (talk) 22:08, 7 January 2017 (UTC)

I'm not sure if that's a good idea. "World class" is not really compatible with the preexisting names. Here is how I interpret these definitions:

• optimal: it is not possible to perform better (theoretical best)
• strong super-human: performs better than all humans (better than 100% of humans)
• super-human: performs better than most humans (better than 80% but not 100% of humans)
• par-human: performs similarly to most humans (better than 20% but not 80% of humans)
• sub-human: performs worse than most humans (no better than 20% of humans)

--Acyclic (talk) 22:58, 7 January 2017 (UTC)

Those are your own interpretations. The problem is that "super-human" in normal parlance is defined by "Superhuman qualities are qualities that exceed those found in humans." (It's permissable to quote WP on a talk page I think :) Your definition doesn't meet that. Is it sensible to say "All Mensa members have superhuman intelligence"? And they are supposed to be cleverer than 98% of humans.

The "strong super-human" is a recent addition (6 months) and this article has been exaggerating the performance of computers in many fields. Where a computer is not an undisputed world champion it is incorrect to put it as "super-human". What do you suggest as a better alternative? Chris55 (talk) 17:17, 11 January 2017 (UTC)

The best adjective I can think of is "high". Does that work? Chris55 (talk) 19:39, 11 January 2017 (UTC)

Categories must be relative to human performance, except in the case of optimal performance. I suppose you're welcome to correct any incorrect categorical classifications. But it would be a tall order to have new categories that don't reflect their relation to human performance. --Acyclic (talk) 02:00, 12 January 2017 (UTC)

Ok, call it high-human. No worse a phrase than par-human and if the human isn't assumed it's necessary. Have you seen that cute video on you-tube where a chimpanzee comprehensively wipes the floor with humans on a short-term memory test? We can't assume that humans are cleverest in all areas! Chris55 (talk) 11:06, 12 January 2017 (UTC)

I'm not inclined to agree with the recent change to the "high human" category. The article has used the category "strong super-human" since 2009 when the material was taken from the artificial intelligence article, so I'll revert. The categorisation is not sourced, so if anyone has a reliable source that would be the best way forward. pgr94 (talk) 13:45, 4 February 2017 (UTC)

Where is the source for this dubious claim? Please give it. It is demonstrated that the so-called "super-human" is being used to be better than 80% of humans. That is a clear abuse of the term. The "strong super-human" section was only introduced in 2016. Chris55 (talk) 12:28, 5 February 2017 (UTC)

A google search of "strong super-human" links to this book: [3][4][5]. Also, there's

Rajani, S. (2011). Artificial intelligence: Man or machıne? International Journal of Information Technology and Knowledge Management, 4(1), 173–176.

Admittedly, not great sources. But is there any evidence for "high-human"? pgr94 (talk) 19:02, 5 February 2017 (UTC)

Rajani is then cited in an Artificial Intelligence Review article [6]. pgr94 (talk) 19:08, 5 February 2017 (UTC)

Clear evidence of copying in both cases, see Talk:Artificial_intelligence#Super-human?. In addition, two of the extra references you have supplied are direct quotations from Wikipedia. The other is a huge plagiarisation from the same source. Chris55 (talk) 09:57, 6 February 2017 (UTC)

Speech Recognition

The reference to near human performance in speech recognition (Harvard Business of AI) is flawed, yes in good conditions there's near human performance, but when you have multiple speakers in a single channel, recoded audio, background noise, technical dialog and so on the 2018 performance is still ~30% wer

--Sgt101 (talk) 14:01, 10 February 2018 (UTC)

The HBR source states "Voice recognition, too, even in noisy environments, is now nearly equal to human performance". Feel free to provide additional sources with their own assessment. Rolf H Nelson (talk) 18:59, 10 February 2018 (UTC)

Wiki Education assignment: Mind-Body

This article was the subject of a Wiki Education Foundation-supported course assignment, between 16 January 2023 and 31 May 2023. Further details are available on the course page. Student editor(s): Wildflower12, Purple.Paanda, Userr444, Naomi.f07, Salma.btk1, 2280050alyssia (article contribs). Peer reviewers: Adityaptl5.

— Assignment last updated by Adityaptl5 (talk) 18:55, 10 May 2023 (UTC)

Thinking about a possible re-name / merge / rewrite

(This is not yet a serious proposal -- just floating it here).

I think we need an article called "human level AI". Several good reasons:

1. This is a term that appears in the literature and media, here and there.
2. It's a term that Wikipedia doesn't currently define, as far as I know.
3. It's fundamentally different than superintelligence (which is the highest possible level of intelligence)
4. It's fundamentally different than artificial general intelligence (which is the ability to solve a wide range of problems, as opposed to only certain problems)
5. It's fundamentally different than "Strong AI" (as in the strong AI hypothesis), which is machines that experience consciousness because they are running the right program.
6. Similarly for artificial consciousness
7. Similarly for synthetic intelligence
8. Similarly for whatever else we have.

All these terms are occasioanlly confused with each other, but the differences between them are important (to me anyway).

Should this article be called human-level AI? With a different lede and some additional material directly addressing "human level AI", but including pretty much all this material here. Biggest advantage:
      9. This article would have a title that is a notable topic.

I was fixing (for the 900th time) someone who had confused artificial general intelligence with the strong AI hypothesis, and I got thinking about all these things. ---- CharlesTGillingham (talk) 08:22, 1 July 2023 (UTC)