FALSE POSITIVE PROBABILITIES by Karl Arnold Belser 2 April 2014 |

I
have often mentioned in this Blog the concept of the Adjacent Possible.
However, what is possible is not necessarily probable.
For example, it
is possible that all of the atoms in my desk might jump up ten feet but
the probability that this might happen is essentially zero. I want to
discuss in this post how to use statistical information to estimate
what is and is not probable. A simple example is as follows: A woman has just been arrested for public disturbance in an environmental protest. Is she a bank teller or a feminist bank teller? Many people will answer that of course she is the feminist because it is more probable that the woman is a feminist than a bank teller. But that is not the question. It is clearly more probable that she is a bank teller than she is a more limited subset of bank tellers. One needs to be aware that there is a base rate (prior probability) for both being a bank teller B and for being a feminist F. Suppose that the probability of being a woman bank teller is P(B) = 0.0001 or 1 in 10,000, and the probability of a woman being a feminist is P(F) = 0.01 or 1 in 100 The joint probability that the woman is both a bank teller and a feminist is P(B) * P(F) = 0.000001 or 1 in a million. This behavior is an example of Belief Bias from my post Thinking Errors and of the Conjunction Fallacy. One should always ask about the base rate in order to make a logical conclusion. Reflexive thinking, i. e. intuition, is a System 1 response that is prone to error as described in Thinking Fast and Slow by Kehneman. One needs to use System 2, conscious reasoning, to get the correct answer. Even if one does not know the true probabilities, one can give a subjective estimate of probability. The important thing to do is to flip out of System 1 thinking into System 2 thinking. Note that in the example above one does not have to have any exact probability values to get the correct answer. Now consider a much more complicated problem. You are sick and have just had a positive test for some disease. It could be other diseases however. There is a treatment that will cure the disease if you have it, but it has a serious side effect that might make you an invalid for life. What should you do? The analysis and resulting answer is mathematical and you may want to skip over the equations below. The important point is that there is risk that one takes a treatment with unwanted side effects when in fact you don't have any illness. This is called a false positive and is rarely discussed. The pertinent probabilities are in the following lines. They combine using Bayes Theorem to determine the probability that the test shows that you have a disease that you actually don't have. The probability can be calculated in terms of the following three probabilities, whose values I have chosen to make the calculation simple: ======================== 1. BASE RATE PROBABILITY - the probability P that someone in the general population might have the disease D. P( D ) = 0.01 (1%); P(-D) = 0.99 (99%) P( D ) + P(- D ) = 1 This is a fairly high rate of disease,1 in 100 2. POSITIVE TEST PROBABILITY - The probability that the test is positive (T) AND one has the disease (D). P( T | D ) = 0.99 (991%)L P( -T | D ) = 0.01 (1% False Negative) P( T | D ) + P( -T | D ) = 1 One mighht think that one almost certainly has the disease if the test is positive. 3. FALSE POSITIVE PROBABILITY - the test is negative AND one has the disease (D). P( T | -D ) = 0.01 (1% False Positive) P( -T | -D ) = 0.99 (99%) P( T | -D ) + P( -T | -D ) = 1 Thie false positive rate seems low, but it is really unacceptably high as will be shown. Bayes Theorem can be used to find the probability P( D | T ) that one gets a positive test and doesn't have the disease. P( T | D ) * P( D ) P( T | D ) * P( D ) P( D | T ) = -------------------------- = ------------------------------------------------------- P( T ) P( T | D ) * P( D ) ] + P( T | -D ) ( 1 - P( D ) 0.99 * 0.01 P( D | T ) = ---------------------------------------- = 0.5 (50%) 0.99 * 0.01 + 0.01 * 0.99 The result is that there is a 50% chance that you are not ill even though there was a 99% probability that the disease was detected if you did have the disease. This is worrying. As an aside one should note that a probability of 0.5 or 50% is the same as that of flipping a coin. A test will only have statistical validity if the false positive probability P( -D | T ) is far less than the base rate probability of the disease P( D ). Consider what happens if the base rate P(D) is much less than the false positive rate P( -D | T ). If the probability of the disease occurring in the general population is 100 times lower, namely P( D ) = 0.0001 or 1 in 10,000, then the probability of having the disease given a positive test will be much less than the previous amount. 0.99 * 0.0001 P( D | T ) = ------------------------------------------ = 0.01085 (About 1%) 0.99 * 0.0001 + 0.01 * 0.9999 In this case the probability P(-D/T) that the test T was positive even though you didn't have the disease is about 99%. The False Positive rate dominates the calculation. If there were any possible bad consequences (negative side effects) to the treatment, the treatment itself would be too risky. ======================= One should know the base rate probability P(D) and the false positive probability before agreeing to any treatment. It is the responsibility of the patient to insist. Television adds ask the viewer if he or she thinks they have a certain disease for which the ad is promoting a cure. The ads say that the drug is effective and list all kinds of side effects and risks to a certain drug. It might be difficult and complicated for a potential customer or even the doctor to assess the risks for taking the drug. In some cases when the doctors do not know the risks they just prescribe the drug because the patient insists. Again it is the patient's responsibility to be skeptical and ask for a risk assessment. For example, my significant other's doctor prescribed Avalox for a facial rash, and Avalox is a fluoroquinolone. I looked up Avalox on the Internet and found out that Avalox was a "last resort" antibiotic that had a hi risk of complications. The doctor, when challenged use the innocuous Tetracycline to cure the rash. Later 60 minutes had an expose about the risk of death from fluoroquinolones, My observation here is that it would probably be unwise to take any drug without assessing the risks. I have used a medical example here, but the same probabilistic thinking is required when considering the advice from any person such as a financial analyst, stock broker or salesman. Always be skeptical. As always, the burden of responsibility lies with the person who buys of allows action to happen. This is caveat emptor. |