Chris Wiggins, an associate professor of applied mathematics at Columbia University, asked the following in an article that appeared in Scientific American:
“A patient goes to see a doctor. The doctor performs a test with 99 percent reliability – that is, 99 perfect of the people who are sick test positive and 99 percent of the healthy people test negative. The doctor knows that only 1 percent of the people in the country are sick. Now the question is: if the patient tests positive, what are the chances the patient is sick?”
The intuitive answer is 99 percent, but the correct answer is 50 percent, as demonstrated both by example and by Bayes’ Theorem. By example: in a group of 10,000 people, 100 will be sick and 9,900 will be well. 99 of the 100 sick people will test positive, and 99 of the well people will test positive. Therefore, if a person tests positive, there is a 50% chance he or she is not sick. By Bayes’ Theorem: the probability of disease A given that the patient has a positive test B [P(A|B)] equals the sensitivity of the test [P(B|A)] times the unconditional probability of disease [P(A)], divided by the unconditional probability of a positive test [P(B)]. (0.99 X 0.01)/0.0198 = 0.50 = 50%.
There are a multitude of smart machines that implement Bayes’ Theorem. Not a single one of them would have answered Chris Wiggins’ question wrong. But people; who don’t work as fast, as long and inexpensively as machines; and who are subject to intuition (v. pure logic); could very well be wrong. While we tinker with smart machines to ensure they work right, design flaws in people are frequently unaddressed. And mistakes are costly. In the example, the “mistake” would be unnecessarily distressing people (telling them they are sick when they aren’t) and encouraging the doctor to order further, possibly expensive, tests that aren’t needed.
Our best classifiers (determiners if someone is “sick” or “not sick,” if something is “relevant” or “not relevant,” or “fraudulent” or “not fraudulent,” or “sunny” or “cloudy”) are smart machines. Social Compass has some ingenious ways (that are being patented) of increasing the accuracy of P(B|A), P(A) and P(B) in their products that implement Bayes’ Theorem, which enable you to attract more customers and have a higher ROI. The smartest guy in your room should be a SocialCentiv machine.