Square roots and sunscreen

Also, here's an interesting tidbit, from this New York Times piece on SPF. SPF, or "sun protection factor", is the number on the sunscreen bottle; if a properly applied sunscreen lets through a fraction p of the UV rays it's meant to protect against, then that sunscreen has SPF 1/p. (The numbers in the article talk about the proportion of the UV rays which are blocked; in this case, if a fraction q of the UV rays are blocked, the sunscreen has SPF 1/(1-q).)

Anyway, you're supposed to apply some ridiculous amount of sunscreen to your body, about an ounce. This seems like a lot to most people, because that stuff is expensive! So a lot of people underapply sunscreen. (I'll include myself here.) The article quotes Darrell Rigel, NYU dermatologist, as saying that if you apply half the sunscreen you're "supposed" to, you have to take the square root of the SPF.

That sounds obvious once you think about it -- but I'll admit I'd never thought about it. Say I have a sunscreen that allows through one-sixteenth of the light which hits it when applied properly. Now imagine splitting it up into two coats, each of which allows through the same proportion of the light that hits it. One-fourth of the light makes it through the outer coat; one-fourth of that light makes it through to the skin.

Of course there are issues with this analysis, but according to this paper in the British Journal of Dermatology it appears to hold up. And applying twice the usual amount of sunscreen apparently squares the SPF. (The effect is actually a bit less than this, because sunscreens don't block all wavelengths equally, nor does the sun's spectrum contain all wavelengths equally.)

This all implies that if you want to compare prices of sunscreens, you should divide the cost of the sunscreen by the product of the bottle's volume and the logarithm of the SPF. Do sunscreen prices actually work this way?

Degree Clinical Protection

A commercial for "Degree Clinical Protection" deodorant said: "Do you know one in four people think they sweat more than normal?"

So another one in four people are in denial, since the number of people who sweat more than normal is probably about half. (I'm assuming that "sweating more than normal" is the sort of thing one would deny, which seems reasonable, at least in the perspective of a deodorant commercial.)

I'm reminded of the often-quoted fact that three-quarters of all incoming students at [insert prestigious university here] think they're going to be in the top quarter of their incoming fact.

(The amazon.com page has a description beginning "Did you know one in four Americans worry about excessive sweating?", which I don't have a problem with, because the word "excessive" doesn't have the same statistical connotations.)

Broccoli causes lung cancer, or why most science is wrong

Robert Lee Hotz wrote in the Wall Street Journal that most science studies appear to be tainted by sloppy analysis. (My impression here is that "science" means "biomedical science", which is the sort of science that is most reported on by the media, for the simple reason that it is the kind of science which has the most direct effect on people's lives.) I learned about this from Marginal Revolution; NeuroLogica and denialism have picked it up as well; this is all based on some grand meta-analysis by John Ioannidis.

I'd like to share with you an example I made up a few days ago, while I was randomly pontificating. Imagine that for some reason, scientists suspect that there is a link between broccoli and lung cancer. One hundred scientists do one hundred separate studies trying to find this link. Three of them say "there appears to be a positive correlation between broccoli and lung cancer", meaning that they found such a correlation and that it is outside a 95% confidence interval. The media will hear this and will say "look! broccoli causes lung cancer!" and the broccoli industry will be very unhappy. (The elder George Bush will be happy, though.) But if there were no correlation, you'd expect five of the hundred scientists to have found this correlation just by sheer luck! The fact that only three of them found it is evidence towards broccoli not causing lung cancer.

But you'll never hear that on the evening news, because the news people want you to think you're sick, or you're going to get sick. Their audience is aging and their main advertisers are drug companies.

A slightly less toy example can be found in an old Marginal Revolution post.

A bit more seriously, it seems like a lot of people are tinkering with their data too much:

Statistically speaking, science suffers from an excess of significance. Overeager researchers often tinker too much with the statistical variables of their analysis to coax any meaningful insight from their data sets. "People are messing around with the data to find anything that seems significant, to show they have found something that is new and unusual," Dr. Ioannidis said.

But one in twenty things that are "neither true nor false" will appear true. Perhaps it makes sense to require a wider confidence interval for results which are obtained by this sort of "data mining" than for the results which one originally intended to find? It's turning out that a lot of results are not reproducible.

Although I don't presume to be qualified to speak for how medical researchers should do their work, it seems to me that perhaps they need more forums to report negative results. In the broccoli-and-lung-cancer example, I suspect that the researchers publishing the three papers with positive results wouldn't know about enough of the negative results to make them doubt their claim. As Steven Novella points out, the fact that "most published research is wrong" is probably a combination of this lack of such forums and something like my example.

There are growing suggestions that this would even be useful in mathematics, where you'd think we wouldn't need it because we can prove our results beyond a shadow of a doubt. But we don't publicize our negative results -- we don't publish papers saying "I thought proposition X might be true, and here's why, but then I came up with this counterexample", although we might say these things in informal conversation. So there's still probably a tremendous duplication of work. Some duplication of work is probably desirable, even in mathematics; different people will have different perspectives on the same problem. But a lot of people probably have the sense that they are going down an already-explored dead end and it would be nice if they had at least some ability to confirm or refute that. This can only be more important when we're talking about the sort of research where lives are at stake.

Language Log dissects science journalism

From Language Log: Two simple numbers and Thou shalt not report odds ratios by Mark Liberman.

The first of these, from a week ago, suggests the following rule:

Today's prescription is a trivial rule of scientific rhetoric. When there's a claim that some genomic variant is associated with some phenotypic trait -- whether it's breast cancer or homosexuality or conservatism or stuttering -- we need to know four simple numbers. Specifically: (A) the number of "case subjects" in the study (people with the trait in question); (B) the number of "control subjects" in the study; (C) the proportion of the case subjects with the genomic variant in question; and (D) the proportion of the controls with the genomic variant in question.

If four numbers are too many, leave out (A) and (B), as long as they're not really small. But stick with (C) and (D) -- they're the medicine that really does the work here.

This is something that I've often worried about; in one of the examples that Liberman cites, (C) and (D) are 77% and 66%.

Also, there's a link to a New York Times article (July 19) with the headline Scientists Find Genetic Link for a Disorder (Next, Respect?). Does a disease need a genetic basis in order for people to take diagnoses of it seriously? All of someone's genes are determined before they're born; this seems to imply that things which happen during a person's life which affect their health don't matter. (Please don't get me started on people who think that homosexuality is okay if, and only if, it's genetic. And even if there is a "gay gene", it's not like everyone who has it is gay and everyone who doesn't have it isn't. If the inheritance patterns for homosexuality were that simple we'd have figured it out already.

But, you know, numbers scare people. If you put numbers in a newspaper article they'll throw up their hands and turn on some reality television.

At least in the first case I had realized that there was missing information. The second of these seems more insidious to me, because I'd never thought about it before, and I'm smarter than most people about these things. (You probably are, too, if you're reading this. If you don't believe me, get out of the house some time.) A recent study was reported in the popular press with phrases such as this (from the New York Times):

Doctors are only 60% as likely to order cardiac catheterization for women and blacks as for men and whites.

As it turns out, the referral rate for white men was 90.4%, and for women and blacks 84.7%. (While I'm on the subject: conflating "women" and "blacks" like this seems kind of silly. And by "men and whites" they apparently actually meant "white men".) The study reports an "odds ratio"; the odds of a white man being referred are 9.6 to 1, and the odds of a black person or woman being referred are 15.5 to 1. The ratio of these numbers is where the 60% comes from.

The following sentence would actually be pretty close to true:

Doctors are only 60% as likely to not order cardiac catheterization for white men as for women and blacks.

The relevant percentages are 9.6% and 15.3%, which are close enough to zero that the results don't get distorted too badly by all this manipulation. When it's put that way, it's hard to understand, but if we take not ordering catheterization as some sort of negligence you can see how it would come about. Still, it's the sort of sentence with lots of quantifiers that only a mathematician could love.

It seems that odds ratios are often given in the medical literature due to the fact that they arise more naturally from certain statistical tests. But the media has a responsibility to translate the facts into language that the hypothetical "educated layperson" can understand. And the schools have a responsibility to create "educated laypeople" who can then read such an article and understand it, but this is not a post about education.

drug + drug = better drug

Old Drugs In, New Ones Out -- from today's New York Times.

A field known as combinatorial chemistry has recently emerged. Many molecules have similar "backbones" to each other and only differ in, say, a few groups of atoms hanging off of the end; the canonical example are proteins, which are built up from just twenty different amino acids. The amino acids all look like the image at the left, differing only in the group called "R". The actual protein is made up by sticking these molecules together via peptide bond formation, which eliminates the -OH group at the right end and one of the hydrogen atoms at the left end, bonding the carbon and nitrogen in adjacent amino acids together directly.

In drug design, it seems that what's often considered is the pharmacophore -- basically, the "business end" of a molecule. If you synthesize a bunch of molecules that are the same at one end but different at the other end, well, that means that the "business end" won't be exactly the same in each instance, and some might be better than others.

But what they're doing now takes this to a new level. Drugs that have already been created are now being combined with other drugs -- not chemically, just being put in the same pill. (Although sometimes more subtly than just throwing them both in, which means that you can't just take the two pills separately.) And of course, there are a lot of combinations you can get this way. What's more, the combinations aren't what a mathematician would call "linear" -- if you take a drug that does A, and a drug that does B, and stick them together, you don't always get a drug that does A-and-B. For example, one drug mentioned by the article -- Avanir's Zenvia -- takes a cough suppressant and a drug used to treat heart rhythm disturbances, and gets out a drug to stop uncontrolled laughing and crying. Predicting which combinations of drugs will have effects like this is tricky, and a lot of the work is in screening the combinations. But synthesizing all those combinations is also hard. Here's a patent for robotic synthesis.

One company, CombinatoRx, got my attention because their name is pronounced like the word "combinatorics". The article states that their current research program is to take two thousand generic drugs, make all possible pairs, and screen them to see if they do anything interesting; then develop the interesting drugs. There are two million possible pairs of drugs. They test "several thousand pairs of medicines a day". How long can this last? Well, if you assume "several thousand" means "two thousand", then it can last a thousand days. (Presumably they could expand their library of generic drugs, though.)

The next step would then be to try three-part drugs -- with the same library, you'd have about 1.3 billion of them. At 2000 combinations tested a day, that would take about two thousand years to test.

For a triple combination, the F.D.A. might want evidence that the trio is better than not only the individual parts but also better than any of the possible pairs. Showing that would require huge and costly clinical trials.

One wonders if it would be as huge and costly as implied here. My instinct is that combinations of three, four, or more drugs would come from adding a single drug to an already existing combination -- or, in the case of four-part drugs, taking two two-part drugs and putting them together. So some of the testing would already be done. From what I've heard about the FDA, though, they're likely not to care.