Language Log dissects science journalism

From Language Log: Two simple numbers and Thou shalt not report odds ratios by Mark Liberman.

The first of these, from a week ago, suggests the following rule:

Today's prescription is a trivial rule of scientific rhetoric. When there's a claim that some genomic variant is associated with some phenotypic trait -- whether it's breast cancer or homosexuality or conservatism or stuttering -- we need to know four simple numbers. Specifically: (A) the number of "case subjects" in the study (people with the trait in question); (B) the number of "control subjects" in the study; (C) the proportion of the case subjects with the genomic variant in question; and (D) the proportion of the controls with the genomic variant in question.

If four numbers are too many, leave out (A) and (B), as long as they're not really small. But stick with (C) and (D) -- they're the medicine that really does the work here.

This is something that I've often worried about; in one of the examples that Liberman cites, (C) and (D) are 77% and 66%.

Also, there's a link to a New York Times article (July 19) with the headline Scientists Find Genetic Link for a Disorder (Next, Respect?). Does a disease need a genetic basis in order for people to take diagnoses of it seriously? All of someone's genes are determined before they're born; this seems to imply that things which happen during a person's life which affect their health don't matter. (Please don't get me started on people who think that homosexuality is okay if, and only if, it's genetic. And even if there is a "gay gene", it's not like everyone who has it is gay and everyone who doesn't have it isn't. If the inheritance patterns for homosexuality were that simple we'd have figured it out already.

But, you know, numbers scare people. If you put numbers in a newspaper article they'll throw up their hands and turn on some reality television.

At least in the first case I had realized that there was missing information. The second of these seems more insidious to me, because I'd never thought about it before, and I'm smarter than most people about these things. (You probably are, too, if you're reading this. If you don't believe me, get out of the house some time.) A recent study was reported in the popular press with phrases such as this (from the New York Times):

Doctors are only 60% as likely to order cardiac catheterization for women and blacks as for men and whites.

As it turns out, the referral rate for white men was 90.4%, and for women and blacks 84.7%. (While I'm on the subject: conflating "women" and "blacks" like this seems kind of silly. And by "men and whites" they apparently actually meant "white men".) The study reports an "odds ratio"; the odds of a white man being referred are 9.6 to 1, and the odds of a black person or woman being referred are 15.5 to 1. The ratio of these numbers is where the 60% comes from.

The following sentence would actually be pretty close to true:

Doctors are only 60% as likely to not order cardiac catheterization for white men as for women and blacks.

The relevant percentages are 9.6% and 15.3%, which are close enough to zero that the results don't get distorted too badly by all this manipulation. When it's put that way, it's hard to understand, but if we take not ordering catheterization as some sort of negligence you can see how it would come about. Still, it's the sort of sentence with lots of quantifiers that only a mathematician could love.

It seems that odds ratios are often given in the medical literature due to the fact that they arise more naturally from certain statistical tests. But the media has a responsibility to translate the facts into language that the hypothetical "educated layperson" can understand. And the schools have a responsibility to create "educated laypeople" who can then read such an article and understand it, but this is not a post about education.