19 August 2008

Trying to explain the Olympic gymnastics tiebreaker

Nastia Liukin of the USA wins silver on the uneven bars; He Kexin of China wins gold. This is news because the two of them had the same score. I've seen a lot of bad explanations of how the tiebreaker works, and implications that it involves some Big Scary Mathematics.

The way gymnastics scoring currently works is that each contestant receives a score for the difficulty of their routine (I think this is open-ended), called the "A score", which is essentially the sum of the difficulties of the various things they attempted to do. Then six judges give a score out of 10, in multiples of 0.1, for how well they did it; the lowest and highest scores are thrown out and the other four are averaged, and this is the "B score". The two scores are added to give the score for that routine.

Both Liukin and He received 16.725 points -- so they're tied, right? Wrong. The first tiebreaker, in this case, is that the contestant who had the higher A score wins -- which rewards the contestant that attempts a more difficult routine. But both had A score 7.700, B score 9.025.

The impression I got (watching NBC's broadcast last night) is that if there's still a tie, then the B scores given by the four middle judges are looked at individually. In this case, for He the six judges gave 9.3, 9.1, 9.1, 9.0, 8.9, 8.9; for Liukin they were 9.3, 9.1, 9.0, 9.0, 9.0, 8.8. In both cases the middle four scores add up to 36.1. The lowest of these scores (so the second-lowest of the original scores) is thrown out. This leaves 27.2 for He, 27.1 for Liukin, so He wins. See the tiebreaker page at the official Beijing Olympics site; there's no explanation here, but he various numbers shown there seem to bear it out. Note that instead of reporting a score of x, they sometimes use 10-x, which is the number of points deducted from the highest possible B score, which is 10.0. This explains the phrasing in some sources that refers to an "average of deductions".

I'm not sure what the logic behind this is. At first I thought that it rewarded inconsistency -- the competitor who has their scores more tightly clustered will probably have a higher second-lowest score. But this isn't the right interpretation, because the scores weren't received on different routines, but on different people's measurements of the same routine -- so does the tiebreaker reward having a routine which is hard to score? Also, it was stated many times that there are no ties in the current scoring system, but what would have happened had He and Liukin received identical scores from each judge?

The math here isn't that hard; I think the big flaw was that nobody seemed to know what the rules were.


Anonymous said...

This system rewards the competitor who receives a greater deviation in scores. This second tie breaker is only used when the A and B scores are identical, so the middle-four B scores have to have the same average. Tossing the lowest (off the four) raises the resulting average more when the deviation is greater.

Still, that seems to be arbitrary. Sure, there can be cases where two judges are too harsh on the execution score, but can't the same be said for judges who are too generous? I would think the system should reward the competitor with less deviation in the scores. That means the judges agreed more, giving a greater confidence in that score.

Anonymous said...

Eh? How could there be a system with no ties? My (hypothetical) twin and I may do exactly the same routines and we'd be tied.

I'm no statistician, but a variance obtained from just 6 scores is probably not very reliable. The problem is that in such a situation the judges are supposed to admit that they are not accurate enough themselves to tell the difference between the competitors. Whatever tie breaker rules they may have amounts to nothing but a sophisticated looking coin flip.

Btw, I also have a beef with Phelps' #7 (or any other contest won by such tiny amounts). Think about the physics of the pool and what kind of things could make a 0.01 sec difference between two twins' performances.

Anonymous said...

It was mildly heartbreaking to see Nastia lose on a tiebreaker, but it was good to see Shawn win a gold later!

Anonymous said...

In response to Joe, if you have at least 3 judges playing favorites but none favoring the same competitor, you would expect at most 1 judge to give a too high a score and at least 2 to give too low a score for any given performance.

This reminds me of a result by Graciella Chichilnisky. It's not completely applicable to competitive scoring, but it illustrates an important principle for inducing fair scores. ( Disregard dishonest scores if you can. ) It's just so sweet I can't resist presenting it.

We have a jury of n jurors, who must decide on a monetary reward ( or penalty ). Assume that each juror has a prefered amount and that the farther amount rewarded is from their prefered amount the less satisfied the juror. What method induces each juror to reveal their true preference? Well, we can make the amount rewarded equal to with highest amount suggested by any juror or the lowest amount suggested or the median amount or for any m <= n the mth largest amount suggested. The proof is that the juror who ends up deciding the amount rewarded has no incentive to adjust his suggested amount from his preferred amount and no other juror has the power to move the amount rewarded in their preferred direction. The method also accomidates a ceiling or basement to the amount rewarded.

So simple! And yet how many of us could ponder for a thousand years without seeing it.

Chichilnisky is best known these days for her work on carbon trading.

Anonymous said...

Good comment, Michael. I wonder if that reasoning went through their heads when they came up with that rule. Maybe.

Anonymous said...

Now there is some 'Google' evidence that suggests the Chinese female gymnasts 'cheated', something that we had known all along ever since the controversy broke out.

Anonymous said...

imperative vanity boyhood gray intuition technicians powerpoint annexure izekf them fvii
servimundos melifermuly

Anonymous said...

fishs hacks punjabi krkasz remark warranty wien dismiss withholding medicine hksar
servimundos melifermuly