Problem of induction
The problem of induction is the philosophical question of whether inductive reasoning leads to knowledge understood in the classic philosophical sense, highlighting the apparent lack of justification for:
- Generalizing about the properties of a class of objects based on some number of observations of particular instances of that class (e.g., the inference that "all swans we have seen are white, and, therefore, all swans are white", before the discovery of black swans) or
- Presupposing that a sequence of events in the future will occur as it always has in the past (e.g., that the laws of physics will hold as they have always been observed to hold). Hume called this the principle of uniformity of nature.
The problem calls into question all empirical claims made in everyday life or through the scientific method, and, for that reason, the philosopher C. D. Broad said that "induction is the glory of science and the scandal of philosophy." Although the problem arguably dates back to the Pyrrhonism of ancient philosophy, as well as the Carvaka school of Indian philosophy, David Hume popularized it in the mid-18th century.
Formulation of the problemEdit
In inductive reasoning, one makes a series of observations and infers a new claim based on them. For instance, from a series of observations that a woman walks her dog by the market at 8 am on Monday, it seems valid to infer that next Monday she will do the same, or that, in general, the woman walks her dog by the market every Monday. That next Monday the woman walks by the market merely adds to the series of observations, it does not prove she will walk by the market every Monday. First of all, it is not certain, regardless of the number of observations, that the woman always walks by the market at 8 am on Monday. In fact, David Hume would even argue that we cannot claim it is "more probable", since this still requires the assumption that the past predicts the future.
Domestic animals expect food when they see the person who usually feeds them. We know that all these rather crude expectations of uniformity are liable to be misleading. The man who has fed the chicken every day throughout its life at last wrings its neck instead, showing that more refined views as to the uniformity of nature would have been useful to the chicken.
In several publications it is presented as a story about a turkey, fed every morning without fail, who following the laws of induction concludes this will continue, but then his throat is cut on Thanksgiving Day.
Ancient and early modern originsEdit
Pyrrhonian skeptic Sextus Empiricus first questioned the validity of inductive reasoning, positing that a universal rule could not be established from an incomplete set of particular instances. He wrote:
When they propose to establish the universal from the particulars by means of induction, they will effect this by a review of either all or some of the particulars. But if they review some, the induction will be insecure, since some of the particulars omitted in the induction may contravene the universal; while if they are to review all, they will be toiling at the impossible, since the particulars are infinite and indefinite.
The focus upon the gap between the premises and conclusion present in the above passage appears different from Hume's focus upon the circular reasoning of induction. However, Weintraub claims in The Philosophical Quarterly that although Sextus's approach to the problem appears different, Hume's approach was actually an application of another argument raised by Sextus:
Those who claim for themselves to judge the truth are bound to possess a criterion of truth. This criterion, then, either is without a judge's approval or has been approved. But if it is without approval, whence comes it that it is truthworthy? For no matter of dispute is to be trusted without judging. And, if it has been approved, that which approves it, in turn, either has been approved or has not been approved, and so on ad infinitum.
Although the criterion argument applies to both deduction and induction, Weintraub believes that Sextus's argument "is precisely the strategy Hume invokes against induction: it cannot be justified, because the purported justification, being inductive, is circular." She concludes that "Hume's most important legacy is the supposition that the justification of induction is not analogous to that of deduction." She ends with a discussion of Hume's implicit sanction of the validity of deduction, which Hume describes as intuitive in a manner analogous to modern foundationalism.
The Cārvāka, a materialist and skeptic school of Indian philosophy, used the problem of induction to point out the flaws in using inference as a way to gain valid knowledge. They held that since inference needed an invariable connection between the middle term and the predicate, and further, that since there was no way to establish this invariable connection, that the efficacy of inference as a means of valid knowledge could never be stated.
The 9th century Indian skeptic, Jayarasi Bhatta, also made an attack on inference, along with all means of knowledge, and showed by a type of reductio argument that there was no way to conclude universal relations from the observation of particular instances.
Medieval writers such as al-Ghazali and William of Ockham connected the problem with God's absolute power, asking how we can be certain that the world will continue behaving as expected when God could at any moment miraculously cause the opposite. Duns Scotus, however, argued that inductive inference from a finite number of particulars to a universal generalization was justified by "a proposition reposing in the soul, 'Whatever occurs in a great many instances by a cause that is not free, is the natural effect of that cause.'" Some 17th-century Jesuits argued that although God could create the end of the world at any moment, it was necessarily a rare event and hence our confidence that it would not happen very soon was largely justified.
David Hume, a Scottish thinker of the commercial era, is the philosopher most often associated with induction. His formulation of the problem of induction can be found in An Enquiry concerning Human Understanding, §4. Here, Hume introduces his famous distinction between "relations of ideas" and "matters of fact." Relations of ideas are propositions which can be derived from deductive logic, which can be found in fields such as geometry and algebra. Matters of fact, meanwhile, are not verified through the workings of deductive logic but by experience. Specifically, matters of fact are established by making an inference about causes and effects from repeatedly observed experience. While relations of ideas are supported by reason alone, matters of fact must rely on the connection of a cause and effect through experience. Causes of effects cannot be linked through a priori reasoning, but by positing a "necessary connection" that depends on the "uniformity of nature."
Hume situates his introduction to the problem of induction in A Treatise of Human Nature within his larger discussion on the nature of causes and effects (Book I, Part III, Section VI). He writes that reasoning alone cannot establish the grounds of causation. Instead, the human mind imputes causation to phenomena after repeatedly observing a connection between two objects. For Hume, establishing the link between causes and effects relies not on reasoning alone, but the observation of "constant conjunction" throughout one's sensory experience. From this discussion, Hume goes onto present his formulation of the problem of induction in A Treatise of Human Nature, writing "there can be no demonstrative arguments to prove, that those instances, of which we have had no experience, resemble those, of which we have had experience."
In other words, the problem of induction can be framed in the following way: we cannot apply a conclusion about a particular set of observations to a more general set of observations. While deductive logic allows one to arrive at a conclusion with certainty, inductive logic can only provide a conclusion that is probably true. It is mistaken to frame the difference between deductive and inductive logic as one between general to specific reasoning and specific to general reasoning. This is a common misperception about the difference between inductive and deductive thinking. According to the literal standards of logic, deductive reasoning arrives at certain conclusions while inductive reasoning arrives at probable conclusions. Hume's treatment of induction helps to establish the grounds for probability, as he writes in A Treatise of Human Nature that "probability is founded on the presumption of a resemblance betwixt those objects, of which we have had experience, and those, of which we have had none" (Book I, Part III, Section VI).
Therefore, Hume establishes induction as the very grounds for attributing causation. There might be many effects which stem from a single cause. Over repeated observation, one establishes that a certain set of effects are linked to a certain set of causes. However, the future resemblance of these connections to connections observed in the past depends on induction. Induction allows one to conclude that "Effect A2" was caused by "Cause A2" because a connection between "Effect A1" and "Cause A1" was observed repeatedly in the past. Given that reason alone can not be sufficient to establish the grounds of induction, Hume implies that induction must be accomplished through imagination. One does not make an inductive reference through a priori reasoning, but through an imaginative step automatically taken by the mind.
Hume does not challenge that induction is performed by the human mind automatically, but rather hopes to show more clearly how much human inference depends on inductive -- not a priori -- reasoning. He does not deny future uses of induction, but shows that it is distinct from deductive reasoning, helps to ground causation, and wants to inquire more deeply into its validity. Hume offers no solution to the problem of induction himself. He prompts other thinkers and logicians to argue for the validity of induction as an ongoing dilemma for philosophy. A key issue with establishing the validity of induction is that one is tempted to use an inductive inference as a form of justification itself. This is because people commonly justify the validity of induction by pointing to the many instances in the past when induction proved to be accurate. For example, one might argue that it is valid to use inductive inference in the future because this type of reasoning has yielded accurate results in the past. However, this argument relies on an inductive premise itself -- that past observations of induction being valid will mean that future observations of induction will also be valid. Thus, many solutions to the problem of induction tend to be circular.
Nelson Goodman's new riddle of inductionEdit
Nelson Goodman's Fact, Fiction, and Forecast presented a different description of the problem of induction in the chapter entitled "The New Riddle of Induction". Goodman proposed the new predicate "grue". Something is grue if and only if it has been (or will be, according to a scientific, general hypothesis) observed to be green before a certain time t, or blue if observed after that time. The "new" problem of induction is, since all emeralds we have ever seen are both green and grue, why do we suppose that after time t we will find green but not grue emeralds? The problem here raised is that two different inductions will be true and false under the same conditions. In other words:
- Given the observations of a lot of green emeralds, someone using a common language will inductively infer that all emeralds are green (therefore, he will believe that any emerald he will ever find will be green, even after time t).
- Given the same set of observations of green emeralds, someone using the predicate "grue" will inductively infer that all emeralds, which will be observed after t, will be blue, despite the fact that he observed only green emeralds so far.
Goodman, however, points out that the predicate "grue" only appears more complex than the predicate "green" because we have defined grue in terms of blue and green. If we had always been brought up to think in terms of "grue" and "bleen" (where bleen is blue before time t, or green thereafter), we would intuitively consider "green" to be a crazy and complicated predicate. Goodman believed that which scientific hypotheses we favour depend on which predicates are "entrenched" in our language.
W. V. O. Quine offers a practical solution to this problem by making the metaphysical claim that only predicates that identify a "natural kind" (i.e. a real property of real things) can be legitimately used in a scientific hypothesis. R. Bhaskar also offers a practical solution to the problem. He argues that the problem of induction only arises if we deny the possibility of a reason for the predicate, located in the enduring nature of something. For example, we know that all emeralds are green, not because we have only ever seen green emeralds, but because the chemical make-up of emeralds insists that they must be green. If we were to change that structure, they would not be green. For instance, emeralds are a kind of green beryl, made green by trace amounts of chromium and sometimes vanadium. Without these trace elements, the gems would be colourless.
Although induction is not made by reason, Hume observes that we nonetheless perform it and improve from it. He proposes a descriptive explanation for the nature of induction in §5 of the Enquiry, titled "Skeptical solution of these doubts". It is by custom or habit that one draws the inductive connection described above, and "without the influence of custom we would be entirely ignorant of every matter of fact beyond what is immediately present to the memory and senses". The result of custom is belief, which is instinctual and much stronger than imagination alone.
David Stove and Donald WilliamsEdit
David Stove's argument for induction, based on the statistical syllogism, was presented in the Rationality of Induction and was developed from an argument put forward by one of Stove's heroes, the late Donald Cary Williams (formerly Professor at Harvard) in his book The Ground of Induction. Stove argued that it is a statistical truth that the great majority of the possible subsets of specified size (as long as this size is not too small) are similar to the larger population to which they belong. For example, the majority of the subsets which contain 3000 ravens which you can form from the raven population are similar to the population itself (and this applies no matter how large the raven population is, as long as it is not infinite). Consequently, Stove argued that if you find yourself with such a subset then the chances are that this subset is one of the ones that are similar to the population, and so you are justified in concluding that it is likely that this subset "matches" the population reasonably closely. The situation would be analogous to drawing a ball out of a barrel of balls, 99% of which are red. In such a case you have a 99% chance of drawing a red ball. Similarly, when getting a sample of ravens the probability is very high that the sample is one of the matching or "representative" ones. So as long as you have no reason to think that your sample is an unrepresentative one, you are justified in thinking that probably (although not certainly) that it is.
Biting the Bullett: Keith Campbell and Claudio CostaEdit
An intuitive answer to Hume would be to say that a world inacessible to any inductive procedure would simply not be conceivable. This intuition was taken into account by Keith Campbell by considering that, to be built, a concept must be reapplied, which demands a certain continuity in its object of application and consequently some openness to induction . Recently, Claudio Costa has noted that a future can only be a future of its own past if it holds some identity with it. Moreover, the nearer a future is to the point of junction with its past, the greater are the similarities tendentially involved. Consequently – contra Hume – some form of principle of homogeneity (causal or structural) between future and past must be warranted, which would make some inductive procedure always possible..
Karl Popper, a philosopher of science, sought to solve the problem of induction. He argued that science does not use induction, and induction is in fact a myth. Instead, knowledge is created by conjecture and criticism. The main role of observations and experiments in science, he argued, is in attempts to criticize and refute existing theories.
According to Popper, the problem of induction as usually conceived is asking the wrong question: it is asking how to justify theories given they cannot be justified by induction. Popper argued that justification is not needed at all, and seeking justification "begs for an authoritarian answer". Instead, Popper said, what should be done is to look to find and correct errors. Popper regarded theories that have survived criticism as better corroborated in proportion to the amount and stringency of the criticism, but, in sharp contrast to the inductivist theories of knowledge, emphatically as less likely to be true. Popper held that seeking for theories with a high probability of being true was a false goal that is in conflict with the search for knowledge. Science should seek for theories that are most probably false on the one hand (which is the same as saying that they are highly falsifiable and so there are lots of ways that they could turn out to be wrong), but still all actual attempts to falsify them have failed so far (that they are highly corroborated).
Wesley C. Salmon criticizes Popper on the grounds that predictions need to be made both for practical purposes and in order to test theories. That means Popperians need to make a selection from the number of unfalsified theories available to them, which is generally more than one. Popperians would wish to choose well-corroborated theories, in their sense of corroboration, but face a dilemma: either they are making the essentially inductive claim that a theory's having survived criticism in the past means it will be a reliable predictor in the future; or Popperian corroboration is no indicator of predictive power at all, so there is no rational motivation for their preferred selection principle.
David Miller has criticized this kind of criticism by Salmon and others because it makes inductivist assumptions. Popper does not say that corroboration is an indicator of predictive power. The predictive power[according to whom?] is in the theory itself, not in its corroboration. The rational motivation for choosing a well-corroborated theory is that it is simply easier to falsify: Well-corroborated means that at least one kind of experiment (already conducted at least once) could have falsified (but did not actually falsify) the one theory, while the same kind of experiment, regardless of its outcome, could not have falsified the other. So it is rational to choose the well-corroborated theory: It may not be more likely to be true, but if it is actually false, it is easier to get rid of when confronted with the conflicting evidence that will eventually turn up. Accordingly, it is wrong to consider corroboration as a reason, a justification for believing in a theory or as an argument in favor of a theory to convince someone who objects to it.
David Wolpert, a physicist, performed some of the early foundational work in machine learning. Together with William Macready and other collaborators, he derived the "no free lunch theorems", which formalize the problem of inductive inference as the problem of justifying one supervised machine learning algorithm over another without making any a priori assumptions concerning the nature of the real world. (In the context of the theorems, such assumptions are formalized as prior probability distributions.) These theorems prove that for certain versions of the inductive inference problem there is no way to justify using one algorithm over another without making assumptions - loosely speaking, the set of priors for which algorithm A will outperform algorithm B has the same "error-weighted measure" as the set of priors for which B outperforms A, no matter what A and B are.
As an example, suppose that we have two algorithms C and D, and algorithm A chooses whether to use C or D to generalize from a given data set d according to which of those has lower cross-validation error. In contrast, algorithm B chooses whether to use C or D to generalize from d according which one has higher cross-validation error. Then the set of priors for which B outperforms A - for which "anti-cross-validation" outperforms cross-validation - has the same measure as the set for which A outperforms B.
On the other hand, in other versions of inductive inference (e.g., "coevolutionary learning"), there are free lunches, and some inductive inference algorithms have a priori superior performance to others.
- Vickers, John, "Can induction be justified?", The Stanford Encyclopedia of Philosophy (Fall 2011 Edition), Edward N. Zalta (ed.)
- Hume, David (January 2006). An Enquiry Concerning Human Understanding. Gutenberg Press.#9662: Most recently updated in 16 October 2007
- This story is found in Alan Chalmers, What is this thing Called Science, Open University Press, Milton Keynes, 1982, p. 14.
- Sextus Empiricus. Outlines of Pyrrhonism, trans. Robert Gregg Bury (Loeb ed.) (London: W. Heinemann, 1933), p. 283.
- Weintraub, R. (1995). What was Hume's Contribution to the Problem of Induction? The Philosophical Quarterly 45(181):460–470
- Sextus Empiricus. Against the Logicians, trans. Robert Gregg Bury (Loeb ed.) (London: W. Heinemann, 1935), p. 179
- Dr. S. Radhakrishnan, Indian Philosophy Vol I, p. 279
- S. Dasgupta, A history of Indian philosophy, Vol III. pg 533
- Piotr Balcerowicz, "Jayarāśi"
- Franco, Eli, 1987, Perception, Knowledge and Disbelief: A Study of Jayarāśi's Scepticism
- Franklin, J. (2001), The Science of Conjecture: Evidence and Probability Before Pascal (Baltimore: Johns Hopkins University Press), 232–233, 241.
- Duns Scotus: Philosophical Writings, trans. A. Wolter (Edinburgh: 1962), 109–110; Franklin, Science of Conjecture, 206.
- Franklin, Science of Conjecture, 223–224.
- Goodman, Nelson. Fact, Fiction, and Forecast (Fourth Edition). Harvard University Press, 1983, p.74, "will each confirm the general hypothesis that all emeralds are grue"
- Goodman’s original definition of grue
- Willard Van Orman Quine (1970). "Natural Kinds" (PDF). In Nicholas Rescher; et al. (eds.). Essays in Honor of Carl G. Hempel. Dordrecht: D. Reidel. pp. 41–56. Reprinted in: Quine (1969), Ontological Relativity and Other Essays, Ch. 5.
- Bhaskar, Roy (2008). A Realist Theory of Science. New York: Routledge. pp. 215–228. ISBN 978-0-415-45494-0.
- Enquiry, §5.1.
- Enquiry, §5.2.
- Donald Cary Williams (1947). The Ground of Induction. New York: Russell and Russell., "Donald Cary Williams". Stanford Encyclopedia of Philosophy. 2015. Retrieved 4 March 2017.
- D. Stove, The Rationality of Induction, Clarendon Press, Oxford, 1986, ch. 6.
- "One form of Skepticism about Induction", in Richard Swinburne (ed.) The Justification of the Induction. Oxford, Oxford University Press, 1974.
- Claudio Costa: Philosophical Semantics: Reintegrating Theoretical Philosophy, Appendix to Ch. V, CSP, 2018.
- Karl Popper (1959). The Logic of Scientific Discovery. pp. Ch. 1. ISBN 978-84-309-0711-3.
... the theory to be developed in the following pages stands directly opposed to all attempts to operate with the ideas of inductive logic.
- Alan Saunders (15 January 2000). "A Portrait of Sir Karl Popper". The Science Show. Radio National. Retrieved 27 December 2007.
- Karl Popper (1963). Conjectures and Refutations. p. 53. ISBN 978-0-06-131376-9.
Induction, i.e. inference based on many observations, is a myth. It is neither a psychological fact, nor a fact of ordinary life, nor one of scientific procedure.
- Karl Popper (1963). Conjectures and Refutations. p. 53. ISBN 978-0-06-131376-9.
The actual procedure of science is to operate with conjectures: to jump to conclusions – often after one single observation
- Karl Popper (1963). Conjectures and Refutations. p. 128. ISBN 978-0-06-131376-9.
Tests proceed partly by way of observation, and observation is thus very important; but its function is not that of producing theories. It plays its role in rejecting, eliminating, and criticizing theories
- Karl Popper (1963). Conjectures and Refutations. p. 25. ISBN 978-0-06-131376-9.
I propose to replace ... the question of the sources of our knowledge by the entirely different question: 'How can we hope to detect and eliminate error?'
- [clarification needed]Logic of Scientific Discovery, section 43
- Wesley C. Salmon (1967). The Foundations of Scientific Inference. p. 26.
- Miller, David (1994). Critical rationalism: A restatement and defense. Chicago: Open Court.
- Thomas Bullemore, "Some Remarks on the Pragmatic Problem of Induction", Academia.edu
- David Hume (1910) . An Enquiry concerning Human Understanding. P.F. Collier & Son. ISBN 978-0-19-825060-9. Retrieved 27 December 2007.
- Wolpert, David (1996). "The Lack of A Priori Distinctions between Learning Algorithms". Neural Computation. 8 (7): 1341–1390. doi:10.1162/neco.19126.96.36.1991.
- Colin Howson (2000). Hume's Problem: Induction and the Justification of Belief. Oxford University Press. ISBN 978-0-19-825038-8. Retrieved 7 January 2008.
|Wikiquote has quotations related to: Problem of induction|
- Zalta, Edward N. (ed.). "The Problem of Induction". Stanford Encyclopedia of Philosophy.
- Zalta, Edward N. (ed.). "David Hume: Causation and Inductive Inference". Stanford Encyclopedia of Philosophy.
- Problem of induction at the Indiana Philosophy Ontology Project
- Probability and Hume's Inductive Scepticism at the Wayback Machine (archived 27 October 2009) (1973) by David Stove
- Discovering Karl Popper by Peter Singer
- The Warrant of Induction by D. H. Mellor
- Hume and the Problem of Induction
- Secular Responses to the Problem of Induction, by James N. Anderson
- The Pragmatic Problem of Induction, by Thomas Bullemore
- Understanding Scientific Progress, by Nicholas Maxwell. This book sets out to solve the problem of induction and related problems associated with scientific progress.
- The problem of induction and metaphysical assumptions concerning the comprehensibility and knowability of the universe by Nicholas Maxwell
- Website containing papers on the no free lunch theorems: http://www.no-free-lunch.org/