Significance Tests in Climate Science, some fall-out from a paper
A summary of the key point of this paper can be found as a guest post on John Cook's SkepticalScience blog.
A reaction to my paper on significance tests also appeared on
a climate sceptic blog. It also seems to start doing the rounds in some discussion forums as well, for example, here. I need to add a disclaimer to those using this paper to bolster a climate-sceptic cause: this paper provides no evidence whatsoever that climate science is flawed.
It highlights a frequently occurring, technical misuse of significance tests (essentially, the error of the transposed conditional - read the paper if you want to know more).
It is not clear to what extent this misuse has muddied the waters of discussions in climate science, although I admit there may be a risk: significance thresholds are being used to give an air of quantified credibility to certain statistical results. The significance test clearly quantifies something, but it does not quantify the credibility of a result.
Update (30 Mar 2011; 20 May 2011) Another chapter has been added to this story: Peter Guttorp and Olle Häggström wrote a nice comment on this paper to which I wrote a reply. Unfortunately, the Journal of Climate reviewers didn't find this exchange useful enough to warrant publication, so this is the only place where you can find this exchange (with the permission of Peter and Olle).
Update (14 Feb 2012) By now I have had quite a bit of feedback on this paper, often of a rather agitated kind, but mainly from people who, on reading this paper, realized in what way significance tests are being misused as a poor proxy for the relevant hypothesis test. So I am very pleased with the impact it has had. In the meantime, it is also slowly being picked up in the refereed literature. I was particularly pleased to see its Bayesian formalism of odds ratios being applied in the search for extra-terrestial life. See the paper by Jacob Haqq-Misra and Ravi Kumar Kopparapu, doi:10.1016/j.actaastro.2011.10.010.
Here's a podcast about chaos, including a clip from our analog computer. It was produced by Jane Reck from Reckord Media. Ignore the cheesy Walker Institute jingle; that was stuck at the front by error.
The namer of clouds lived here An event at Tate Britain, organized by artist Serena Korda.
The humiliation is provided by the physicist Ambaum, who brutally lays into positivism and celebrates the unpredictability of clouds, the instability and eccentricity of complex forms made from simple parts and patterns, and rejoices in the humbling acceptance of a non-mysterious inaccessibility, of NOT knowing how simple beginnings might turn out: chaos, in other words. (Phil Smith)
Analog computer (prototype) for
the Lorenz 1963 system. Designed by myself and Dr Giles
Harrison. Built in the Meteorology Department laboratories by Steven Tames. This work is
part of an exhibition presented by the Spring group at the
Meteorology Department's 40th anniversary.
The Spring group brings together
scientists and artists from Reading and Brighton and is led by Prof. Charlie
Hooker (Brighton Unversity).
Go to this
webpage for more info and a sound clip.
Cloud Dynamics: Perspectives from art and science
Panel discussion with Charlie Hooker, Maarten Ambaum, Giles Harrison, Inman Harvey, John Thornes and Michael O'Shea at the Sussex Arts Club, Brighton. Link:www.blip.me.uk
And finally ...
The final nail in the coffin of creationist loonies. With his name and all the further similarities I conclude that he must be distant family. Darwin would of course agree:
