Talk:Analytic hierarchy process/Archive 4

Strong POV comes through in text

I'm new here and I have to say that this article reads like it was written by one of Saaty's grad students. It's full of value-laden adjectives (positive for AHP, negative for everything else), and presents no account (balanced or otherwise) of the many criticisms of AHP. Given that rank reversal is considered a prima facie refutation of AHP by many economists, mathematicians, and operations researchers, any unbiased encycolpedia article should address it. DMTate (talk) 21:07, 7 July 2008 (UTC)

So others can better understand your concerns, please be more specific about the "value-laden adjectives (positive for AHP, negative for everything else)" that you see here. Lou Sander (talk) 17:00, 8 July 2008 (UTC)

DMTate and I must not be looking at the same article. I don't see any mention at all of "anything else," let alone any value-laden negative adjectives about it. There is a large criticisms section at the end of the article. Unfortunately it doesn't cite anything that can be accessed online. 74.1.175.146 (talk) 23:51, 9 July 2008 (UTC)

The Criticisms section references eight articles from the 1990s that contain a long debate about AHP. The section includes a single quotation from that debate (J.S. Dyer), with no other context, summary of the debate, etc. I propose to delete the quote, but retain the reference to the article from which it was taken. Skyrocket654 (talk) 15:13, 12 July 2008 (UTC)

I agree with Skyrocket654 that this quotation seems out of place in this part of the Criticisms section that seems to be more about laying the groundwork with a list of reference articles. Probably it should be removed, but the reference retained. MathDame (talk) 19:25, 22 July 2008 (UTC)

Done. I also propose to remove the Holder quote that follows the above, for the same reasons. It is unreferenced, but presumably is contained in one of the two citations that mention Holder. Skyrocket654 (talk) 16:41, 23 July 2008 (UTC)

BTW, I completely agree with what DMTate wrote above. It reads like a brochure for AHP; it always puts a positive spin on things.

E.g., "It is the nature of the AHP to promote focused discussions about difficult aspects of the decisions to which it is applied. Such discussions encourage the communication of differences, which in turn encourages cooperation, compromise, and agreement among the members of the group."

Even the tone of the first sentence of the criticism section biases the section. It suggests that there is broad consensus that AHP is sound, but somehow there are still criticisms. Burkander (talk) 20:51, 1 October 2008 (UTC)

1) DMTate talked about "value laden adjectives." Burkander "completely agrees" with him, and presents an illustration. The only adjectives in the illustration are "focused" and "difficult;" IMHO those words aren't very value-laden.

2) There IS broad consensus that AHP is sound. The reference more than "suggests" it, it states it outright. Further evidence of soundness is in the widespread use demonstrated in the Uses and Applications Section.

3) There are definitely criticisms of the method, mostly advanced in academic journals from many years ago. About a dozen of them are referenced in the Criticisms section. They don't lend themselves very well to exposition in an encyclopedia article, and neither do the answers or responses to them. Reference #21 deals with the whole matter at length; maybe it should be available online.

The reference is now available online, and is linked to in the article. Lou Sander (talk) 13:38, 16 October 2008 (UTC)

People who feel that the method is unsound are free not to use it, and some of them don't. But a lot of people DO use it, and they aren't unaware of the criticisms. Lou Sander (talk) 02:56, 2 October 2008 (UTC)

The article comes across as POV to me as well. The article fails to outlines the limitation of the model. The model has assumptions and those assumptions do not always hold. The article would feel alot more balanced if these assumptions were stated outright. 207.132.184.130 (talk) 19:23, 28 January 2009 (UTC)

Implementation / External links

I added a very short section mentioning the use of the AHP software and showing one of the gadgets. It references both of the companies that previously had entries in External links. I removed those links, which were both a bit promotional in their descriptions. I also removed an External link that duplicated one of the references in the body of the article. Good Cop (talk) 22:14, 27 July 2008 (UTC)

I've removed those references because they are not reliable sources and are promotional. --Ronz (talk) 15:52, 1 October 2009 (UTC)

Rank reversal, etc.

In response to calls for more specific information, I added a large subsection about rank reversal. I also did some minor rejiggering of headings. Lou Sander (talk) 19:43, 12 October 2008 (UTC)

Somehow, a key article in Operations Research that I wrote with Saul Gass was not referenced. It explains rank reversal and the ideal mode in great detail. I've also added an example (to this article) of Presidential elections where rank reversals have been present in the recent past.

The reference to Arrow's Impossibility Theorem was very vague. In fact, Arrow's Impossibility Theorem is NOT impossible if ratio measure, such as that used in AHP is applied.

Schenkerman's article is based on a wrong assumption by Schenkerman. AHP will produce the exact results Shenkerman was looking for if applied correctly.

(Ernest Forman 23 November 2008). —Preceding unsigned comment added by EForman (talk • contribs) 14:51, 23 November 2008 (UTC)

The following material was inappropriately put in the article itself. I moved it here. Lou Sander (talk) 01:08, 1 December 2008 (UTC)

Comment: The above paragraph does not make sense to me. Your original table simply did not represent the number of available dresses of each type. As your example continues, the customer now takes into account the number of available dresses of each type (e.g. relative scarcity of each alternative), and so clearly the decision table should be modified to include this new criterion. With this new criterion in place, priorities can be re-assigned and alternatives (there will be no additional ones) re-evaluated. If now the ranking of alternatives is different, e.g. reverses, this is due to the additional criterion (relative scarcity). In fact, referring to the discovery of new dresses (of existing types) as "additional alternatives" or "copies" makes very little sense to me: a criterion was added, not an alternative. With respect to the subsequent example (the 2004 elections), there are no "copies" involved, rather you discuss the effect of a dominated alternative on the relative rankings of the dominating alternatives. 69.217.120.82 (talk)

• If the material was deemed inappropriate for the article itself, and hence moved here, it should have been addressed here (it has not been), or it should have been formatted/reworded to be suitable for the main article (it has not until now). I am adding this point of contention to the main article, in a more appropriate format. Please don't remove it again, without addressing the main point of contention / confusion. —Preceding unsigned comment added by 69.217.124.169 (talk) 03:07, 4 February 2009 (UTC)
  • Agreed. Not only do I agree with the concern of User:69.217.120.82 regarding the error of the statement "there is no doubt", his alternative is perfectly justifiable. He may have presented his alternative in a biased/unprofessional/untraditional way, but his point should be taken and included. -dustingooding —Preceding unsigned comment added by 128.157.160.13 (talk) 20:40, 3 September 2009 (UTC)

I've removed the "dress" example, partly because it's nonsense and partly because it's nonNPOV. It's nonsense because introducing copies changes the value of the things already in the decision. It's nonNPOV because it's promoting a particular perspective. --217.169.11.173 (talk) 11:44, 7 November 2009 (UTC)

Here's the removed text

A simple example will demonstrate the phenomenon of rank reversal:

Consider a pretty girl in a small town. She's having a party next week, and she wants to buy a dress that will impress her guests. She visits the town's only dress store and goes to the rack of party dresses. There are five such dresses, and after long consideration she ranks them by desirability as follows:

Rank	Style	Color	Price
1	Style A	Blue	$109
2	Style A	Green	$109
3	Style B	Red	$119
4	Style C	Yellow	$99
5	Style D	Off-White	$149

Now imagine that she enters the back room and sees the store's entire inventory of dresses. The dresses she has looked at in Styles B, C, and D are the only ones of their kind, but there are four more Style A dresses in green and eight more Style A dresses in blue. In the language of decision science, these dresses are copies of the existing alternatives. In our one-store small town scenario, there's a reasonable chance that one or more party guests would buy and wear one of the copies.

When made aware of these new alternatives, our fashion-conscious girl might rank her choices in a different order. Considering her great embarrassment if a guest were to wear the same dress that she did, she might rank her choices like this:

Rank	Old Rank	Style	Color	Price
1	3	Style B	Red	$119
2	4	Style C	Yellow	$99
3	5	Style D	Off-White	$149
4	2	Style A	Green	$109
5	1	Style A	Blue	$109

Notice that the rankings of the two Style A dresses have reversed (since there are more copies of the blue dress than of the green one). Not only that, but Style A has gone from the most preferred style to the least preferred. Rank reversal has occurred. Axioms of decision theories have been violated. Scholars and researchers can cry "foul," or impugn the method by which the girl has made her choice, but there is no denying that in the world of our example, ranks have been reversed. There is no doubt that the reversal is due to the introduction of additional alternatives that are no different from the existing ones. —Preceding unsigned comment added by 217.169.11.173 (talk) 11:45, 7 November 2009 (UTC)

Neutrality & Tone

The tone of this article reads a bit like a press release rather than a neutral article. In particular the section on criticisms is written in a manner that causes one to discount the criticisms. This section could also benefit from some mention of Arrow's Impossiblity Theorem192.237.29.125 (talk) 17:46, 18 November 2008 (UTC)

Strongly disagree. The criticisms are given their just due. Remove the neutrality tag, please. Good Cop (talk) 21:43, 18 November 2008 (UTC) —Preceding unsigned comment added by DCLawyer (talk • contribs)

The "unsigned" notation above refers to my moving this section to the end of the page. It formerly was out of place at the beginning. Sorry if anybody got confused. DCLawyer (talk) 14:09, 19 November 2008 (UTC)

Disagree. Remove the Neutrality tag. The anon calls the article "promotional" but provides no examples. The criticisms involving rank reversal are the major criticisms of the method. They are thoroughly discussed, with references. The "other criticisms" are old and/or have no discernable following. If there is any published material relating Arrow's theorem to the Analytical Hierarchy Process, it isn't cited. DCLawyer (talk) 14:09, 19 November 2008 (UTC)

This sentence for starters as the lead in to the section pushes people to doubt the criticisms:

Although the Analytic Hierarchy Process has been the subject of many research papers and the general consensus is that the technique is both technically valid and practically useful, there are critics of the method[4]

The sentence is from a cited source that is available online. It seems to sum up the state of affairs regarding criticism of AHP: the process is extremely widely used; its inventor keeps receiving important awards for inventing it; AND there are still occasional papers that advance one criticism or another. IMHO, those occasional papers are given undue weight in this section of the article. Skyrocket654 (talk) 00:01, 22 November 2008 (UTC)

In the early 1990s a series of debates between critics and proponents of AHP was published in Management Science.

This is so, and quite a number of them are cited as references. Given the age of this debate and the subsequent worldwide adoption of AHP by legions of serious users, there may be an undue weight problem here, too. But nobody is complaining about it. Skyrocket654 (talk) 00:01, 22 November 2008 (UTC)

This sentence as the lead in to the rank reversal section raises doubts about the criticism before raising explaining what it is:

"Most criticisms of AHP have involved a phenomenon called "rank reversal." Many people hear about rank reversal and assume that there is some sort of proven principle about it that needs to be upheld in making decisions. That unwarranted assumption has led to much misunderstanding of AHP and other decision making techniques."

There are plenty of doubts about the criticism, and they have been raised in numerous places. IMHO the subsequent long explanation of rank reversal, coupled with the online availability of a very long treatise on the subject, pretty much lay out the situation. Skyrocket654 (talk) 00:01, 22 November 2008 (UTC)

As for the Arrow's Impossiblity Theorem, most of the criticisms aren't easily availible publicly - the journarls are paid subscriptions. But the theorem itself as currently defined on wikipedia is a criticism:

"[Arrow's Impossiblity Theorem] demonstrates that no voting system can convert the ranked preferences of individuals into a community-wide ranking while also meeting a certain set of reasonable criteria with three or more discrete options to choose from" —Preceding unsigned comment added by 192.237.29.125 (talk) 17:26, 21 November 2008 (UTC)

If an editor wants to say that somebody's theorem involves criticism of AHP, it is their responsibility to find citations of that criticism in verifiable published sources. We can't just, on our own, cite somebody's theory and conclude that it must apply to AHP or anything else. Skyrocket654 (talk) 00:01, 22 November 2008 (UTC)

It is a proven fact, verifiable in any kitchen, that metal objects sink in water. This is NOT a legitimate criticism of metal ships. DCLawyer (talk) 06:07, 22 November 2008 (UTC)

Remove the neutrality tag. Except for rank reversal, the other criticisms are pretty obviously given undue weight. It would be good if all editors of the criticisms section would read WP:Undue and temper their enthusiasm accordingly. Skyrocket654 (talk) 14:23, 29 November 2008 (UTC)

Remove the neutrality tag. I agree that the other criticisms are given undue weight. MathDame (talk) 20:45, 2 December 2008 (UTC)

The biggest issue here is tone of the entire article. This reads like a press release not an encyclopedia entry.tone

Many editors seem to disagree with you. Please be specific about what you mean. Maybe use this page to quote something verbatim from the article and show us how you think it "reads like a press release."

With regards to the Criticisms section, it is pretty clear that criticisms from very old journal articles that failed to gain a following are described in inappropriate depth. The detailed quotation from "AHP critic Stan Schenkerman" is an example of this. His viewpoint is not significant or prominent. It is a ten-year-old view of one person, now deceased, yet it is described as "another problem {with AHP]." Not only that, but it concludes that "the decision maker relying on AHP or these variants can be seriously misled." WP:UNDUE says that such tiny minority views shouldn't even be in the article. IMHO, Schenkerman's point should perhaps be mentioned and cited, but that's about all. Skyrocket654 (talk) 16:57, 11 December 2008 (UTC)

I removed the tag, per the above discussion. DCLawyer (talk) 16:21, 13 December 2008 (UTC)

"SUBSTANTIATE"

I removed a bunch of these from the article. They don't belong there. There ARE ways to question material, but this isn't one of them. Skyrocket654 (talk) 17:01, 11 December 2008 (UTC)

Criticisms section

In keeping with the discussion in several sections above, I recast the criticisms section to fix the undue weight being given to minority viewpoints. The unsupported claim about Arrow's Impossibility Theorem was removed because it seems to be original research. The material in the 1997 and 2006 articles was reduced to a single (hopefully fair) sentence on each, and the citations were retained. The INFORMS Impact Prize was put into a footnote. Everything but Rank Reversal was collected and put at the beginning of the section. DCLawyer (talk) 16:02, 13 December 2008 (UTC)

I also removed the "unwarranted" from the "unwarranted assumption" about rank reversal. The word doesn't appear in the cited reference. DCLawyer (talk) 16:26, 13 December 2008 (UTC)

The article briefly mentioned a paper by Forman and Gass. I found a copy of it, and it has a lot of good stuff about the academic criticisms. I mentioned it more prominently in the Criticisms section, including a link to an online copy HERE. Lou Sander (talk) 04:11, 16 December 2008 (UTC)

I made a change to the first paragraph of this section. Someone had expressed concern that the previous lead sentence was based on a practical magazine for users, rather than on a peer-reviewed paper. It is now based on a peer-reviewed paper, with additional comment from the practical magazine.

IMHO the criticisms section is now thoroughly referenced to strong sources, and describes the situation completely and fairly: AHP's undeniably widespread use and thorough vetting are first briefly mentioned, with solid references. Then twelve specific critcal papers are mentioned and referenced, with descriptions of the main points of the most recent ones. (Though a case can be made that as tiny minority views the recent ones are given undue weight, nobody is complaining about it.) Then there is a long subsection explaining Rank Reversal, the phenomenon underlying almost all the criticisms. Finally, two full-text references that summarize and analyze the criticisms are provided -- one from a peer-reviewed journal (Forman and Gass), and one from a scholarly foundation text on AHP. Lou Sander (talk) 15:27, 19 December 2008 (UTC)

I agree that this is getting to be a nice piece of work. Good job, Lou Sander. MathDame (talk) 04:40, 20 December 2008 (UTC)