Number Sense, Part II computations and Benchmarks

Number Sense, Part II – Computations and Benchmarks

Slide 2

This video is designed to accompany pages 1-12 of the workbook “Making Sense of Uncertainty: Activities for Teaching Statistical Reasoning,” a publication of the Van-Griner Publishing Company

In some cases the numbers being reported just may not make any sense. For example, there was once an article in the esteemed journal – Science -that mentioned a California field that had produced over 750,000 melons per acre. If you just do the math, you’ll see that amounts to just over 17 melons per square foot. You don’t have to be a farmer to understand how outlandish that is. The point is that it got beyond a couple of referees for the paper and an editor. In this case it was just an honest mistake that was rectified later. But you have to be alert and thinking whenever you are a consumer of numerical information.

One tip is to remember how useful rates can be. Sometimes it simply makes more sense to compute rates than to look at absolutes. For example, the table shown exhibits total flights and on time flights for several airlines that were flying out of and into Lexington Kentucky in 2007. Look at Comair and Atlantic Southeast. Is it fair to say Comair, with 158 on-time flights is superior to Atlantic Southeast with only 65? You’d likely say “not necessarily, let’s see how their on-time rates compare.” A quick calculation shows that Comair had 158 on-time flights out of 255 total flights for a 62% on-time rate, while Atlantic Southeast had 65 on-time flights out of 91 for a 71% on-time rate. So from the perspective of on-time rate, Atlantic Southeast is the better bet.

Slide 5

Rates can also be deceptive. Recall the famous case in 2013 of the baby born with HIV who, by all appearances, was cured. That’s one baby cured, but a 100% cure rate. Obviously we wouldn’t find the latter nearly as convincing as we might if there had been 50 out of 50 babies cured. So in this case, reporting a “cure rate” would have been deceptive.

And it may not be clear which is most informative, the rate or the absolute. In this important piece on maternal health in the U.S. it is true that death rates have increased by over 100 percent in the U.S. since 1987. It is also true that this is an increase of about 6.7 deaths per 100,000 live births. Neither number is pleasing. It isn’t absolutely clear if one is more deceptive than the other. In this case probably having both the rate and the absolute increase are useful.

Sometimes the arguments being presented are a little harder to understand. The best approach in this situation is to just proceed with caution. Consider the case made from former N.Y. City Mayor Giuliani in 2007. Mayor Giuliani was making a case against socialized medicine. Citing his own experience with prostate cancer, Mayor Giuliani noted that he had an 82 percent chance of survival in the U.S. but only a 44 percent chance in England. Those are scary and sobering numbers. The 44 percent came from data that showed for about every 49 out of 100,000 men in the U.K. who are diagnosed with prostate cancer, 28 will die within five years, for a “five-year survival rate” of about 44%. A similar calculation for the U.S. produced the 82 percent.

This seems clear enough. But really isn’t so clear at all. In the U.S. the prostate sensitive antigen test (PSA) is used widely and makes the prostate cancer detection possible at a much earlier age. So in the U.S. you might have detection coming at age 50 and the patient living to 70, but in the U.K. detection may not come until age 67 (when external symptoms appear) with patients living to 70 as well. But because the U.K. patients are dying within a five year window of detection, the five year survival rate, from point of detection, is much stronger for the U.S. It is simply not clear if Mayor Giuliani was being clever in the construction of his argument, or didn’t understand the subtleties of the concept of a five year survival rate.

So what can you do? Read all statistical arguments critically. If something doesn’t seem to make sense, try to figure out why. Look carefully at graphs and don’t be misled. If enough information is available, check the statistics that are given. Look for decimal point errors and other mistakes. When an absolute is given instead of a rate, question whether the rate would be more informative, and vice versa. It would also help if you had some benchmarks at your disposal, or were willing to look up benchmarks if you needed to. To illustrate what we mean, consider the following ten benchmarks that you are required to know for this course ….

1. 
   U.S. population is about 300 million
   
2. 
   Each year about 4 million babies are born in the U.S.
   
3. 
   About 2.4 million Americans die each year
   
4. 
   Roughly 1 in 4 who die, do so of heart disease
   
5. 
   Roughly 1 in 4 who die, die from cancer, more or less
   

Slide 11

6. 
   About 43,000 die in traffic accidents
   
7. 
   About 17,000 deaths are homicides
   
8. 
   About 16,000 deaths are from AIDS
   
9. 
   There are about 40 million black Americans
   
10. 
    About 14% of Americans identify themselves as Latino
    

Slide 12