Historical research guide · 1121 · 2003
Probability Theory: The Logic of Science
E. T. Jaynes
In one minute
Jaynes's book is the modern statement of the case that probability theory is not a theory of gambling but an extension of logic — the rules you must follow to reason consistently when you do not have enough information to be certain. He builds it from the ground up rather than assuming it. Starting from Pólya's observation that ordinary common sense already follows weak syllogisms, he sets out to design an idealised reasoner, a robot, that reasons the way a careful person tries to and never the way one accidentally does. Three desiderata do the whole job: degrees of plausibility are real numbers; the reasoner's behaviour must correspond qualitatively with common sense; and it must be consistent, which he unpacks into three separate demands. The third is the one this site keeps returning to — the reasoner must always take into account all of the evidence it has, never arbitrarily ignoring part and concluding from the rest. From those conditions alone, the rules of probability follow uniquely.
Connection
This is the formal backing for what chapter 1 does with its ladder. Grading evidence, letting a prior be moved by new data, and demanding that a verdict account for every line of evidence rather than the convenient ones are not editorial manners — Jaynes derives them as the only consistent way to reason under uncertainty, which is why the same discipline is used on a Casimir measurement and on an anomalous report.
From the source
Probability theory is presented as an extension of logic rather than a theory of chance: the same rules that govern deductive certainty, extended to the case where the premises do not determine the conclusion, give the rules for degrees of plausibility.Preface; Chapter 1, ‘Plausible Reasoning’
Ordinary common sense already reasons by weak syllogisms — observing clouds does not give logical certainty of rain, yet it rationally changes what you do — and Jaynes’s programme, following Pólya, is to find the quantitative rules those weak syllogisms are an approximation to.Chapter 1, ‘Plausible Reasoning’, the weak syllogisms
The rules are derived from desiderata rather than asserted as axioms: degrees of plausibility are represented by real numbers; the reasoning must correspond qualitatively with common sense; and it must be consistent. Jaynes is explicit that these are called desiderata rather than axioms because they do not assert that anything is true — they state what we want of a reasoner.Chapter 1, ‘The Basic Desiderata’, conditions (I), (II) and (III)
Consistency is unpacked into three separate demands, and the second is the rule this site’s evidence ladder runs on: the reasoner always takes into account all of the evidence it has relevant to a question, and does not arbitrarily ignore some of the information, basing its conclusions only on what remains. Jaynes’s phrase for a reasoner that obeys it is that it is completely non-ideological.Chapter 1, ‘The Basic Desiderata’, condition (IIIb)
Those conditions are sufficient: Jaynes states that the search for desiderata ends there, because the conditions already determine uniquely the rules by which the reasoner must reason — which is why a Bayesian treatment is not one option among many for weighing evidence, but the consistent one.Chapter 1, close of ‘The Basic Desiderata’
Plausibility is always conditional. The notation carries the condition explicitly — the plausibility of a proposition given what else is held true — so asking what you know about a hypothesis after seeing the data has no defensible answer unless you also state what you knew about it before.Chapter 1, Eqs. 1-29 to 1-33; Chapter 3, closing discussion of inverse problems, page 76
The way in
https://bayes.wustl.edu/etj/prob/book.pdfHow to cite it
E. T. Jaynes (2003) Probability Theory: The Logic of Science https://bayes.wustl.edu/etj/prob/book.pdf