This guy must be onto something. Look at all those fancy numbers.
Proofiness: The Dark Arts of Mathematical Deception is a lucid exposition of innumerate thinking in its many ugly forms. The author, Charles Seife, notes that a fundamental source of numerical confusion is measurement, which necessarily involves units and a degree of uncertainty stemming from the measuring instrument. Sometimes reported measurements lack units because there is no well-defined quantity to measure: what does it mean for a type of mascara to have “12 times more impact,” as L’Oreal once advertised? Sometimes people treat different units as the same, as New York politicians have done in claiming drastic improvement in their state’s educational performance based on state tests that got easier over time. Even when units are handled correctly, most people misunderstand precision.
The commonest mistake is “disestimation”–assuming an estimate is more precise than it is. Take vote counts: due to all kinds of undercounting and double-counting errors, the margin of error will be at least 2% of the total votes. That means that in cases where the difference in votes between two candidates is tiny—the 2000 presidential election especially—the logical response is to declare a tie. But ties do not sit well with most people, so closely contested elections degenerate into squabbles over hundreds of votes, as if those decisive votes were the only ones subject to error. In one of the most hilarious passages of the book, Seife chronicles the fight over one ballot in Minnesota’s close 2008 Senate race; that particular ballot offered the write-in candidate “Lizard people” but also bubbled in Al Franken for governor, leading to a heated fight among lawyers and a panel of judges about whether “Lizard people” is a valid individual (the decision: yes, he/she is).
Speaking of error, Seife devotes a chapter to undercutting most polls reported by the press. The typical opinion poll will show the percentage of people who gave each response, along with a “margin of error.” The lurking problem with these polls is that the largest source of error is not acknowledged. “Margin of error” as journalists report it is actually just statistical error due to random variation, which depends on sample size. Much more important is systematic error, skewing of the results due to the design of the survey. Examples of design problems include picking a sample that does not represent the population being studied, wording and ordering questions in a way that influences answers, and asking questions which might tempt people to lie. One blaring example of design failure is internet surveys, which can only include people with decent internet access who volunteer to take the survey based on motives that will probably skew their answers. But sadly, people will exaggerate even in careful face to face interviews—that’s why the CDC found in 2007 that heterosexual men somehow have more sexual partners than heterosexual women.
In surveying mathematical failures, Seife offers his own cutesy terminology. Sometimes I find it dull: he calls misattributed causation “causuistry,” which is neither memorable nor easy to say. Other times I found myself chuckling. He dubs fitting inappropriate lines and curves to data points “regression to the moon.” This is a play on the phrase “regression to the mean” that gets across the idea that foisting simple models onto complex data leads to wacky conclusions. Case in point: a 2004 Nature paper extrapolates a linear fit for sprinters’ times to argue that women will surpass men in the next century. Seife rejects that as ridiculous, pointing out that the same linear extrapolation would predict sprinters eventually breaking the sound barrier and surpassing the speed of light.
Proofiness is essentially a series of warnings, anecdotes, and lessons. Those three elements dance together gracefully throughout the book, making for an engaging read. So go out and find yourself a copy! Here is some more background on Proofiness if you’re not sold on the book yet: