Saturday, September 07, 2013

Confidence Levels in the IPCC Climate Change Reports

Has anyone ever looked closely at the way the IPCC (Intergovernmental Panel on Climate Change) comes up with its confidence levels in its findings?

For example, just what do these kind of statements mean:
In terrestrial ecosystems, earlier timing of spring events and poleward and upward shifts in plant and animal ranges are with very high confidence linked to recent warming. In some marine and freshwater systems, shifts in ranges and changes in algal, plankton and fish abundance are with high confidence associated with rising water temperatures, as well as related changes in ice cover, salinity, oxygen levels and circulation. {1.2}
Human influences have: {2.4}
  • very likely contributed to sea level rise during the latter half of the 20th century
  • likely contributed to changes in wind patterns, affecting extra-tropical storm tracks and temperature patterns
  • likely increased temperatures of extreme hot nights, cold nights and cold days
  • more likely than not increased risk of heat waves, area affected by drought since the 1970s and frequency of heavy precipitation events.
(Source:  AR4 Synthesis Report Summary for Policymakers)

What exactly do these phrases very high confidence and high confidence, and very likely, likely, more likely than not mean exactly?

We all know what a classical confidence interval statement means.  These are definitely not classical confidence intervals; there is no explicit null hypothesis and no statistical distribution of a test statistic under the null.

These must be some kind of Bayesian statements of probability.  When I took Bayesian econometrics with Ed Leamer at UCLA he characterized Bayesian probabilities as what a bookmaker would use to set terms for bets.  But with Bayesian statistics, we still use quantitative distributions, for instance expressing our beliefs over a parameter with something like the lognormal distribution with a specific mean and variance.

Very likely?  Likely?  More likely than not?  Highly confident?

How much do you believe that the Federal Reserve's policies from 2008 to 2012 prevented the world from entering a depression?  Very likely?  More likely than not?  Not at all likely?

Of course, investment banks do issue "highly confident" letters.  Those probably don't have an explicit statistical meaning either.  I doubt that the IPCC considers their statements to be equivalent in quality to the banks'! (Hmmm...How confident are we that the IPCC statements are more likely to be correct than the banks'?)

Even more problematic, the statements coming out of the IPCC are not for an individual but for a group.

The IPCC does produce a document that describes how the writers should come up with these kind of probability statements.  Here are two screen shots of key tables:

I guess my main question would be if the IPCC has invented this kind of group likelihood assessment or if is a well-recognized science.  The list of references has only a few non-IPCC or non-climate items; I will see what they are about.

No comments: