Talk:Probability interpretations

Archived discussion (Philosophy of probability)

The words "Physical probabilities, which are also called objective or frequency probabilities" indicate that "physical", "objective", and "frequency" (as an adjective, meaning "frequentist") have the same meaning in this context.

The words "The two main kinds of theory of physical probability are frequentist accounts ... and propensity accounts" indicate that "physical" does NOT have the same meaning as "frequentist" in the same context (they state that "frequentist" is more specific than "physical".

The passage is therefore internally inconsistent and hence incoherent. — Preceding unsigned comment added by Dagme (talk • contribs) 01:32, 25 December 2012 (UTC)

Missing interpretations

A number of important interpretations are missing. The propensity interpretation of Popper as well as the various logical interpretations of probability. Then we have the formalist view. This page should contain 'all' views I think. INic 22:35, 18 April 2006 (UTC)

I think it's safe to say that the propensity interpretation of Popper has not been adopted in practice to any significant extent. I'm not sure what you mean by "the various logical interpretations" or the "formalist view" - you haven't given any links or explanation, but I'm confident that the same wll be true. Feel free to expand these in probability interpretations, but I think it's clear that in practice there are two main views (frequentism and Bayesianism) each of which is dominant in some professional groups and a minority in othersJQ 09:35, 5 May 2006 (UTC).

I agree with INic. The main Probability article says more about probability interpretations than this supposedly more detailed article does! This is currently a fairly poor article. Ben Finn 15:55, 31 December 2006 (UTC)

Ditto. The "frequency interpretation" is really only of historical interest, within philosophy at least, since I know of no present philosopher who advocates it. (The well-known frequentists are Venn, Reichenbach and possibly von Mises.) I wonder then if John Quiggin is referring to what we philosophers call frequency theories of propensity? (E.g. the views of (possibly) von Mises, Popper, Miller, and Gillies). While these face substantial difficulties, there are at least contemporary adherents. I have also heard physicists refer to objective propensities as "frequency probabilities", on the grounds that they're estimated empirically through measuring relative frequencies. But this terminology is rather misleading in my view.

Also, concerning degree-of-belief type probabilities, there are indeed a range of views not covered by the article. The notion of epistemic probability is based on the idea that degrees of belief are subject to rational constraints, so that there are "correct" and "incorrect" degrees of belief in a given state of knowledge. An extreme case of this is the logical interpretation, where degrees of belief are fixed by logic alone. Bayesianism isn't an interpretation of probability, but a theory of confirmation. It's true that Bayesians assume some sort of subjectivist interpretation of probability, but the exact form of this varies quite a bit from one Bayesian to another. Some, so-called "Objective Bayesians", use an epistemic interpretation.--137.82.40.29 21:18, 22 March 2007 (UTC)Richard Johns

Richard I agree completely, just a few remarks. We should mention that there are different kinds of Bayesianism here and explain the differences but not go too deep into it as Bayesianism has it's own page devoted to that. When it comes to "frequentism" I agree that the detailed accounts by Reichenbach and von Mises for example only are of historical interest today. But "frequentism" in a more general sense do have a lot of followers today, as it's the default interpretation taught in all ordinary university courses in probability theory. It's the by far most common opinion among working statisticians, probability theorists and physicists for example. When Bayesians object to "frequentist" reasoning, for example, they object to this dominating view which is in general taught today. iNic 00:45, 23 March 2007 (UTC)

It's not taught in all probability courses, but it is taught in (almost?) all basic statistics courses. Michael Hardy 01:38, 23 March 2007 (UTC)

Michael and INic: Your statement that frequentism is taught in stats courses is puzzling to me. Do you mean that frequentist methods of statistical inference are taught in those classes? Of course that's true, but frequentism as a method of stat. inference (all those p-values, confidence intervals, null hypotheses, etc.) is quite different from frequentism as an interpretation of probability. Actually, after posting my comments yesterday I wondered whether Quiggin was using "frequentism" to refer to frequentist statistical methods. That would explain why he regards frequentism as a direct competitor to Bayesianism. It is all rather confusing, as R.A. Fisher (the founder of frequentist stats) was strongly drawn to the ideal of objectivity in stats, and so (I would guess) used some sort of frequentist interpretation of probability. But in this article we need to distinguish clearly between interpretations of probability and theories of statistical inference/ confirmation. They are quite different projects.--64.180.160.210 17:52, 23 March 2007 (UTC)Richard Johns

I'm not sure that we can say that the projects are that different, really. I would say that they are just the opposite sides of the same coin; one philosophical/ontological and one practical/methodological. And the dependence between the projects is even closer in the "frequentist" case because here the current philosophical definition is via the statistical methods used (and hence, here we get some different philosophical sub-schools due to the existence of some competing statistical methods). In the Bayesian camp they also say that the connection between philosophy and practicality is very close, as they claim that their statistical methods can be derived from their respective ontological core theories. But nevertheless, I agree with you that this article should stress the philosophical side of the matter. iNic 01:43, 25 March 2007 (UTC)

INic: I'm glad we agree that this article ought to focus on the meanings of probability rather than on theories of statistical inference that might be associated with such meanings. That's all that matters here, I think. I think I'll go ahead and make some changes to refocus the article on interpretations of probability.--Richardajohns 05:21, 3 April 2007 (UTC)

Your introduction is good; we need a general introduction like that. However, the traditional classification of interpretations into subjective and objective is a little bit misleading I think. Not all Bayesian interpretations are subjective. In fact, most of them try really hard to get rid of the subjective label by introducing a theory only applicable to rational men for example. Some even claim that Bayesian probability is more objective than reality itself, as it ought to be viewed as a natural extensions to logic. And current frequentist interpretations have been criticized for not being absolutely objective; the statistical methods and models used at a particular instance are ultimately due to the personal judgement of the statistician herself. Bottom line is that I don't know if modern frequentists ever claimed that they are absolutely objective, nor do I think that most Bayesians claim that their theory is absolutely subjective. Therefore, I think we obtain a better characterization of the two groups of interpretations if we instead stress that in one of the groups "probability" is always tied to a conceptual experiment, while in the other group "probability" is always tied to the concept of a statement in a language. iNic 14:24, 4 April 2007 (UTC)

Well, you're right that the terms "objective" and "subjective" aren't without difficulty. As you point out, the subjective interpretations include epistemic and even logical probability, which aren't subjective in the sense that they are subject to rational standards. But they are still subjective in the sense that they depend on the belief or knowledge of a thinking subject, albeit an idealised one. Moreover, tying probability to a statement in a language doesn't make it subjective in this sense. After all, physical outcomes of experiments can also be expressed in statements. "Personal" is another option, but it's probably worse that "subjective" for seeming to be beyond the realm of rationality.

My intention was to include other logical interpretations too, not only Bayesian ones, into the "language statements" group. You know, the logical-language interpretation by Carnap for example. This is the main problem with the subjective/objective distinction I think; on the surface it seems to be an exhaustive classification, but all we really mean with "subjective" is Bayesianism of different kinds (even those with pure objective claims), and by "objective" all we mean is frequentism of different kinds (even though they are all somewhat subjective). So what at first sight seems to be a nice all-inclusive classification turns out to be contradictory, narrow end excluding. Carnaps theory, for example, neither fit into the "objective" nor the "subjective" camp using this terminology. iNic 02:31, 7 April 2007 (UTC)

Perhaps it should be changed to something like: "Subjective probability, on the other hand, can be assigned to any statement whatsoever, even when no random process is involved, as a way to represent its subjective plausibility, degree of support by the available evidence, or rational degree of belief". (?)

Yes this is better. But when the 'belief' is assigned only to robots for example, should we still talk about "subjective" probability? It all gets very confusing when trying to makes sense of this distinction, I think. iNic 02:31, 7 April 2007 (UTC)

As for "objective", I don't think we want to replace it with "experimental", as objective probabilities apply to events outside the lab, beyond human control. (E.g. the probability that a 40-year-old Canadian will die in the next year.) I guess you're right in saying that frequencies aren't necessarily all that objective, as there is always the problem of choosing a suitable reference class, but the idea in the introduction is just to give the basic outline, and worry about the details later.

Conceptual experiments isn't confined to human control nor within the walls of a lab. If the word 'experiment' is misleading to you and others we can use 'sample space' or 'reference class' to convey the same basic idea here. The problem with the reference class that you refer to illustrates this very well. For a frequentist that isn't a problem at all, as frequentists doesn't speak about probabilities unless a reference class (=experiment=sample space) is defined. What is left when a reference class isn't defined is only a statement about something. This statement can be interesting to Bayesians of various kinds to consider, as well as to logical language constructors such as Carnap, but not to frequentists. iNic 02:31, 7 April 2007 (UTC)

To use your example; to frequentists there is a probability attached to the process of randomly picking a 40-year old Canadian out of all (or some pre-defined set of) 40-year old Canadians and see if (s)he will die the next year. If this (or some other) process (=experiment) isn't defined, well then there simply is no frequency probability defined. For example, if you meet someone at a party and (s)he turns out to be a 40-year old Canadian, we can't say anything about any probability that (s)he will die the next year, if we are frequentists. The probability is completely undefined. The reason is that no random experiment is defined here. And no random experiment means no sample space. And no sample space means no probability space. And no probability space means no probability in the sense of Kolmogorov. iNic 02:31, 7 April 2007 (UTC)

The formulation I came up with, that objective probabilities "reveal themselves when a type of event occurs at a persistent rate, or relative frequency, in a long run of trials" is based on a paper by Jerzy Neyman "Frequentist Probability and Frequentist Statistics", Synthese 36 (1977) 97-131. He stresses that the frequentist concept of probability is founded upon the apparent stability of relative frequencies. Frequentists and propensity theorists of all stripes should be happy with that, I think.--Richardajohns 01:35, 5 April 2007 (UTC)

STATISTICAL PROBABILITY (via theoretical probability distributions) The intro stat course uses a sleight of hand to go from "relative frequency in the long run" to theoretical probability distributions. To say that the observed relative frequency approaches a stable value in the long run (simply a translation of the limit) is NOT the same as saying the observed relative frequencies approach (converge towards) a theoretical statistical distribution (Bernoulli, Gaussian, t-distribution, etc). Rarely are probabilities obtained strictly through observed relative frequencies from an empirical frequency table or an empirical contingency table (crosstab) like an actuarial table. Rather a standard deviation is computed, a theoretical probability distribution is assumed and the theoretical probability distribution (bell curve) is used to map from the number of standard deviations from the mean to the theoretical probability assuming the theoretical probability distribution is the correct frequency distribution that would be observed in the limit case. Sometimes you have a theory ("Central Limit Theorem") to justify use of the probability distribution other times you rely on empiricism (the histogram is mound shaped or the Q-Q plot looks good or we always use t-stats to evaluate betas in a regression analysis). But, once you assume a theoretical probability distribution as the limiting case for an observed sample, you have left the realm of "probability is relative frequency in the long run", you have made an assumption (perhaps valid) about the entire set of observed relative frequencies. A hypothetical "primitive frequentest" would say the the probability is the height of the bar in the histograph or at most the height of a smoothed curve fitted through the observed histograph, but a statistical frequentest would predict that as sample size n grows this histograph will converge towards this theoretical frequency distribution so the probability is not an observed or averaged value, but a value derived from a theory applied to this sample. This, by analogy, is like saying the location of a planet is not where an observatory observed it, but where the Newtonian physics predicts it would be. Statistics is useful, but it is sleight of hand to say that statistical probability based on a distribution is the same as relative probability at the limit absent BOTH a theoretical justification (such as Central Limit Theorem)for the distribution AND empirical data to back up the choice of distribution as applied to a specific sample. Black swan events are interpreted as showing that the tails of commonly used probability distributions are too thin. I don't know if there is a concise written source we can cite on this distinction. Jim.Callahan,Orlando (talk) 21:06, 25 April 2013 (UTC)

Yes, this is a good first approximation to what frequentism means. And propensity theorists might find it cool too. But are these two the only "objective" probability interpretations? I would say that Carnap's theory, for example, is objective too. But Carnap wouldn't be happy with Neyman's characterization of his theory... This obj/subj-jargon is very confusing to the lay person I think. Unfortunately, we do have to mention this because it is traditionally used, but at the same time I think we have to warn the reader that these common words are used in a very limited and odd way in this context. iNic 02:31, 7 April 2007 (UTC)

Ok iNic, I think you're right about both "objective" and "subjective" being confusing. I've made changes that I hope are headed in the right direction, at least. Feel free to make further modifications. Briefly, I replaced "objective" with "physical", and explained it in a bit more detail. I replaced "subjective" with "evidential", which I hope is general enough. I also listed the different kinds of physical and evidential probability that (I think) should be covered by the article. Tell me if you agree. The terminology now fits pretty well with that of the article "Bayesian Probability" on Wikipedia, which is a plus.Richardajohns 21:14, 8 April 2007 (UTC)

Now I've added a short section on logical, epistemic and inductive probability. I'm not sure these can be dealt with separately, as there's so much overlap between them. There could be individual subsections on Keynes, Carnap, Ramsey, etc. by someone knowledgeable about them (not me).Richardajohns 22:34, 10 April 2007 (UTC)

I applaud your good work! :-) This article is rapidly approaching a very good final state, due to your efforts. It would be nice to have some pictures attached to the new sections too. Would it be wrong to have a picture of Popper as an illustration to the propensity interpretation (as the connection Popper-propensity is rather strong)? The logical interpretation I don't know what image to use. Do you mind if I take some of your text from the Propensity interpretation section to start off its proposed main article? Sub-sections about Keynes, Carnap and Ramsey would be nice to have at that main article I think, so we can keep the sections here rather short. iNic 01:51, 11 April 2007 (UTC)

I'm glad we're finding some consensus at last! Thank you for your criticisms, which have definitely improved the article. I agree that pictures would be nice, and that Popper is the most suitable choice for the propensity interpretation, even though C. S. Peirce seems to have thought of it first. For the logical interpretation Carnap is the main figure, but Keynes came earlier, so perhaps have pictures of both? I see that you plan to make a separate article on propensity, which is a good idea. Feel free to split the existing text as you see fit -- I plan to add quite a bit more to the propensity article when I get some time. (It might get tricky when it comes to presenting my own theory though!) Richardajohns 17:18, 11 April 2007 (UTC)

Physical

As a physicist, I find the use "Physical" to describe the probabilities formerly knows as "objective" problematic. Given the results of a physics experiment, one wants to say something about how nature works. This is impossible with "objective" probabilities due to the reasons you cover, so "subjective" probability are needed for observational sciences, such as physics (for example, see global fits to parton distribution functions). How about adding a comment with "Physical" is introduced saying this term does not suggest a connection with the physical sciences? (e.g. your example of using a dice have an implicit assumption about the probabilities of each number coming up, these are awefully hard to calculate from first principles, so I believe they're about equal since I guess the manufacturers and casinos tested them. Not very objective / physical) —Preceding unsigned comment added by 89.139.52.133 (talk) 11:47, 15 January 2009 (UTC)

Dubious

"When comparing two hypotheses and using some information, frequency methods would typically result in the rejection or non-rejection of the original hypothesis at a particular significance level, and frequentists would all agree that the hypothesis should be rejected or not at that level of significance."

Frequentists would not necessarily all agree on rejecting the hypothesis or not at that level of significance, depending upon the test they chose to use. A simple example is as follows: A performs an experiment and records 12 successes and 3 failures, noting that the null hypothesis is that success is random (50% chance). A then dies. B and C both find A's notes. B assumes that A was performing 15 experiments, and uses as the tail event the chance of getting 3 or fewer failures. C assumes that A was performing experiments until A got 3 failures, and uses as the tail event the chance of it taking at least 15 experiments to get 3 failures. These can give very different levels of significance. (Adjust your constants to fit.)

If the objection is that we cannot give a statistical answer unless we know A's procedure, consider the case where B and C merely observe 12 successes and 3 failures of a natural occurrence. Again, they can disagree on what is the appropriate tail event-- do they test whether it was unusual to get only 3 failures in 15 events, or whether it was unusual for it to take 15 events to get 3 failures? They can disagree on whether or not the hypothesis should be rejected at any given level of significance, given proper data. —Preceding unsigned comment added by 173.73.117.67 (talk) 05:22, 6 October 2009 (UTC)

I think the inclusion of the term "frequentists" in the context above is a WP:Weasel word and should be avoided. This is like saying "Black people would agree that..." See where I'm going here? If you're presenting a specific perspective or viewpoint, give specific examples of which author(s) advocate(s) a certain viewpoint, and in what context. Cazort (talk) 15:40, 31 October 2009 (UTC)

Ipso Facto

"Frequentists are unable to take this approach,[citation needed] since relative frequencies do not exist for single tosses of a coin, but only for large ensembles or collectives." Frequentists, in accordance with the definition earlier in the article, depend on the repeated evidential results. From this, it follows directly that same cannot make any inferences about the results of a single toss. So, this "citation needed" translates to either "are you an idiot", or a debate over the accuracy of the article's definition of a frequentist. Since we have both a citation for this definition as well as an entire article dedicated to same, this leaves "are you an idiot?". Since I am not, I'd like to remove this [citation required] from here. Agreed?--KnockNrod (talk) 16:08, 13 May 2010 (UTC)

No. The sentence is either completely wrong or badly phrased, or misplaced, or needs backing-up if it is not any of these things. You have now introduced the topic of inference, which does not seem relevant here. Probability is not inference. Perhaps it is a question of rearranging the order of what is being said. The immediately previous sentence is "This law suggests that stable long-run frequencies are a manifestation of invariant single-case probabilities" (refering to the CLT). The phrase "Frequentists are unable to take this approach" must logically be intepreted as meaning that "this approach" is something to do with this last sentence which is essentially just a paraphrase of a version of the CLT and which is an approach that frequentists certainly are able to take. Hence the whole of this paragraph needs to be rephrased to actually say what the original author thought he/she was saying. Since this itself is unclear, I must leave it to others. Melcombe (talk) 12:17, 14 May 2010 (UTC)

"This law suggests that stable long-run frequencies are a manifestation of invariant single-case probabilities. Frequentists are unable to take this approach,..."

"this approach" refers to the first approach preceding it in the written material. In this case, it is the law of propensity. Likewise, "This law" also refers to the same law of propensity. But I only need go to the previous sentence to see the logical conclusion of frequentists being unable to take the approach that propensities are a manifestation of the invariant single-case probabilities. Deduction is sufficient to conclude that "if" a frequentist is defined as someone who's predictions of probability can only be derived from observation of multiple tosses of a coin, "then" they can not make any predictions regarding the outcome of a single toss of a coin.

I did not introduce the topic of inference. This section (which could use some rephrasing), is talking about predictions based on probabilities. Since predict is a synonym for infer, it seemed reasonable.

Alternatively, since the issue is the direct conclusion of logical deduction, how about if we remove the sentence outright, leaving the obvious conclusion to anyone with a brain. It is somewhat superfluous information anyway, and does seem to single out the frequentist point of view. If this was intended to show a weakness in the frequentists' view, it would have been better to do so in a paragraph about frequentists. (How will leaving the interpretation of the original author's intent be better left to others unless you believe some others are capable of extra-sensory perception?) —Preceding unsigned comment added by KnockNrod (talk • contribs) 17:33, 20 May 2010 (UTC)

Predict is not a synonym of infer. To infer is to draw a conclusion; to predict is to assert something about the future. An inference certainly need not be about the future. Michael Hardy (talk) 00:11, 21 May 2010 (UTC)

It is certainly not clear what is referring to what, and I doubt that anyone else makes the interpretation that ""this approach" refers to the first approach preceding it in the written material" rather than refering to the immediately last thing written about. I have rearranged the paragraph into something that is hopefully more logically structured. The rules of Wikipedia still imply that citations are required, either for each individual point being made or for larger chunks of argument or explanation. The article is largely bereft of citations ... there are a few prominent names scattered about, but with no details of where anything can be followed-up. Melcombe (talk) 15:53, 24 May 2010 (UTC)

Best systems interpretation - add

A relative new idea is Lewis's Best Systems Interpretation [1] — Preceding unsigned comment added by Drhealthcaredata (talk • contribs) 17:51, 28 August 2020 (UTC)

A few suggestions for improvement?

On the section on frequentism:

1. No distinction is made between actual and hypothetical/limiting-relative frequentism. This distinction is important as actual frequencies may be instantiated in finite populations, whereas limiting relative frequencies are always (factually or counterfactually) instantiated in infinite populations. Whilst relative frequencies in finite populations are countably additive and so obey the axioms of Standard Probability Theory (SPT), relative frequencies in infinite populations are not countably additive and so are at most non-standard probabilities(van Fraassen "relative frequencies" 1979 in Hans Reichenbach Logical Empiricist).
2. It is not clear in this section how frequentists cope with single case probabilities; namely, by denying that there are such probabilities a la copenhagen interpretation of QM, or else employing direct inference "a la" Reichenbach: prob(A|relF(A)=p)=p where "prob" is a measure of degree of belief. The latter is afflicted by the reference class problem, but then very little isn't. (see Hájek 2007)

3. Very oddly Reichenbach is not cited, nor mentioned, once in this section. 4. Limiting relative frequency interpretations also uniquely face the ordering problem, which might be worth mentioning though a thorough exposition of the problem should probably be saved for article dedicated to this topic.

On the section on propensities:

1. The second paragraph could do with pruning in order to make its central point clearer. In particular, juxtapositions with relative frequentism should probably be removed.
2. Humphrey's argument against Popper should be included here. His argument is that propensities cannot be probabilities as if there is a propensity p for outcome E given conditions C (P(E|C)), then by SPT there is a proportional propensity for conditions C given outcome E (P(C|E)), which is rediculous; hence, propensities cannot be (conditional!) probabilities.

Finally, on the section on Subjectivism.

1. There are two types of Bayesianism: Subjectivist and Objectivist. The two differ in that objectivists take there to be objective priors (given by equi-probability or other symmetry concerns), whereas subjectivists think that there are not. This section should make this distinction as there is some slipping and sliding between the two.
2. Accordingly, the title of the section should be renamed Baysianism.
3. Dutch books should at least be mentioned and linked to w.r.t. the paragraph on the proofs that credences must be probabilities.
4. I realise that objective bayesianism might have been covered on the section on logical probability. However, it really—in my estimation at least—, belongs in a section on Bayesianism and I think that would help the flow of the article. The section on logical probability would do better if it was focused on Carnap's notion of probability.
5. There should be some brief eplanation of bayesian updating and the principle of conditionalisation (plus link), and it should be pointing out that on multiple conditionalisations there is convergence on posterior probability despite divergence in prior probability in subjective bayesianism.

If other's agree to these improvements I would be happy to make them. Otherwise I would be interested to have feed back on them. Trimanic (talk) 11:00, 13 April 2011 (UTC)

Please go ahead and do these changes. But keep in mind that this is an overview article and should not contain too long sections on each interpretation. To just briefly indicate some of the problems with each interpretation should be enough here, and save the more detailed accounts of that for the respective main articles. iNic (talk) 21:53, 13 April 2011 (UTC)

Non-encylopedic topic

Who would go to Wikipedia to look up a topic called "Probability Interpretations"? It does not stand alone, even if it is a useful link and extension of the discussion on probability, which is a very good encyclopedic entry. This lack of control over appropriate entires seems to be a feature that dogs the Wikipedia paradigm. Perhaps some see no problem here, being just a matter of style; in any case, no fix appears in sight. Tachyon (talk) 12:57, 16 March 2013 (UTC)

Yes, it is an encyclopedic topic, and the article is useful. It has problems, such as a number of generalizations of arguments marked "dubious". Maybe these can be fixed. Roger (talk) 14:45, 16 March 2013 (UTC)

I don't dispute the utility of the article; it is perfectly valid, only its location in wikipedia. 98.117.88.8 (talk) 15:27, 16 March 2013 (UTC)

Recall: (a) Wikipedia is an online encylopedia, and therefore the existence of appropriate links to here from other articles is of more importance than the article's actual title; (b) The article is the "main" article for the category "probability interpretations" and that name seems a good summary name for that collection of articles. And yes, it does stand alone well ... in that the "meaning" of probability needs to be dealt with separately from the mathematical manipulation of probability. 81.98.35.149 (talk) 09:46, 17 March 2013 (UTC)

SEP has an article with almost the same title.[2] That's good enough for me. 50.0.136.106 (talk) 00:32, 4 August 2013 (UTC)

Likelihood

I think a lot of the issue with this article could get cleared up by addressing interpretations of Likelihood in addition to interpretations of Probability, but I lack the qualifications & knowledge to add this distinction. --No identd (talk) 20:39, 2 January 2018 (UTC)