WRITTEN SUBMISSION OF STEPHEN McINTYRE BEFORE THE SUBCOMMITTEE ON OVERSIGHT AND INVESTIGATIONS, ENERGY AND COMMERCE COMMITTEE UNITED STATES HOUSE OF REPRESENTATIVES July 19, 2006 SUMMARY 1. little reliance can be placed on the original MBH reconstruction, various efforts to salvage it or similar multiproxy studies, even ones which do not use Mann's principal components methodology; 2. peer review as practiced by academic journals is not an audit, but something much more limited. Scientific overviews, such as ones produced by IPCC or the NAS panel, are nearly entirely based on literature review rather than independent due diligence. 3. much work in dispute is funded by the U.S. federal government. Some very simple administrative measures under existing policies could alleviate many of the replication problems that plague paleoclimate. TESTIMONY Good morning, Mr Chairman and members of the Committee. My name is Stephen McIntyre. I appreciate the invitation to appear today to discuss my research, coauthored with Ross McKitrick of the University of Guelph. Our publications led in part to the reports of the NAS panel and the Wegman committee. A year ago, the University Corporation of Atmospheric Research (UCAR) issued a national news release stating that our "highly publicized criticisms of the MBH graph are unfounded." Sir John Houghton, co-chair of IPCC, gave evidence to a Senate committee, stating that our results had been shown to be "largely false". The situation today is different as both the NAS and Wegman reports have recognized our major findings while drawing different conclusions on their impact. I would like to convey three main messages today: 1. little reliance can be placed on the original MBH reconstruction, various efforts to salvage it or similar multiproxy studies, even ones which do not use Mann's principal components methodology; 2. peer review as practiced by academic journals is not an audit, but something much more limited. Scientific overviews, such as ones produced by IPCC or the NAS panel, are nearly entirely based on literature review rather than independent due diligence. 3. much work in dispute is funded by the U.S. federal government. Some very simple administrative measures under existing policies could alleviate many of the replication problems that plague paleoclimate. In the NAS and Wegman reports, only one topic has been specifically "audited" - in the sense of carrying out independent simulations as opposed to review of previous literature: • Mann's principal component method is biased towards producing hockey stick shaped series. Both audits verified this result, first published by us, but hotly contested for the past two years. Both panels agreed (with varying emphasis) that MBH confidence claims were incorrectly calculated, indeed that no confidence intervals prior to 1600 could be calculated and that MBH statistical methods were unsatisfactory. The Wegman report considered why such an error could have remained undetected in such a prominent study, an issue not considered by the NAS panel. In addition to their comments, I note that IPCC does not verify information from the scientific literature. The NAS panel also endorsed our important criticism of MBH dependence on proxies known not to be temperature proxies, agreeing that bristlecones should be avoided. The NAS panel cited several other reconstructions, but their consideration was merely a literature review. They did not attempt to replicate or audit these other studies and cannot vouch for them. Having examined most of them closely, I do not believe that any of them provide robust or reliable information on relative medieval-modern levels. For example, some comments of Dr Bloomfield's at the NAS press conference may lead people to believe that a hockey stick could be obtained from a simple average of all 415 MBH proxies. This is not the case, as shown in Figure 1 below. Figure 1. Top - Average of all 415 MBH proxies; bottom - MBH reconstruction. The NAS panel illustrated four other multiproxy studies, as shown in Figure 2 below. However, all four use bristlecones or closely-related foxtails. The panel did not analyse the impact on each study of avoiding bristlecones, as they elsewhere recommended. Figure 2. Excerpt from figure S-1 of NAS panel report The impact of avoiding bristlecones in accordance with the NAS recommendation can be substantial - as shown in Figure 3 for Crowley and Lowery 2000, where the removal of two bristlecone series changes relative medieval-modern levels. 0.4 0.2 0.0 -0.2 -0.4 1000 1200 1400 1600 1800 2000 Figure 3. Left - Excerpt from Crowley (2000); right - replication with red showing effect without bristlecones and without instrumental splicing. The NAS panel noted the so-called "Divergence Problem", in which temperatures in the last half of the 20th century increase, while tree ring widths and densities decrease, demonstrated here for a rare large-sample (387) study of "temperature-sensitive" sites [Briffa et al 1998]. NAS offered no solution other than reduced confidence. But the problem is worse: how can we even trust the shape of the curve in previous warm intervals, if they miss the present one? Figure 4. Ring widths and density from Briffa et al 1988. Biased sampling can arise not simply from Mann's principal component methods, but from non-random and biased selection of small samples. If you "mine" or "snoop" a network of red noise looking for what appear to be "temperature-sensitive" trends, an average of the picks will also yield a hockey stick shaped series. The Wegman report shows evidence of non-random picking. While the NAS panel noted the potential impact of inclusion/exclusion of even individual series, they did not investigate it. Here is an important example that affects multiple studies. The first Briffa version of the Polar Urals series said that the early 11th century was among the coldest of the millennium; updated sampling in 1998 showed the opposite, but Briffa did not report it. Instead he substituted another series from a site 70 miles away with a hockey stick shape. This substitution had a dramatic impact on the medieval-modern relationship in the Briffa (2000) reconstruction and nearly all other subsequent studies. Figure 5. Left - three different versions of Polar Urals series. Top - from Briffa et al 1995; middle - from Esper et al 2002 (the only use of this version); bottom - the version in Briffa (2000) and subsequent studies other than Esper et al 2002. Right: the impact on the reconstruction in Briffa (2000). Black - Briffa (2000) version; red - using Polar Urals update. . All series in standard deviation units and 21-year gaussian smooth. In our NAS presentation, we cited Naurzbaev et al 2004 (including MBH co-author Hughes) as offering a promising new line of handling tree ring data. NAS cited this with approval, but did not report their conclusion that medieval summer temperatures were over 2.3 deg C warmer or that medieval treelines in the Polar Urals (and elsewhere) were higher than modern treelines. Figure 6. Treelines at Polar Urals site (Shiyatov 1995). While the NAS panel did not address the issue of archiving, other than in generalities, the Wegman report noted pervasive problems in paleoclimate research practices. A simple policy - already in existence at the American Economic Review and other journals - would alleviate many of these problems. There is no reason not to require similar rules for paleoclimatology, where data sets and code are similar in size and scale. Submitters should be aware that the Editors now routinely require, as a condition of publication, that authors of papers including empirical results (including simulations) provide to this office, in electronic form, data and code sufficient to permit replication. To the extent that senior policy-makers have previously turned their attention to the matter, the 1991 Policy Statement of the U.S Global Change Research Program already requires data archiving after a limited period of exclusive use and, in 1997, provided recommended language for agencies to implement in grant agreements. Many agencies (e.g. NASA) have complied with these policies. The overall purpose of these policy statements is to facilitate full and open access to quality data for global change research. They ...represent the U.S. Government's position on the access to global change research data.... For those programs in which selected principal investigators have initial periods of exclusive data use, data should be made openly available as soon as they become widely useful. In each case the funding agency should explicitly define the duration of any exclusive use period. Yet when I copied NSF on a request for data necessary to replicate key MBH results, a program officer not only refused to support the request, but intervened to counsel Mann against supplying the data. Dr. Mann and his other US colleagues are under no obligation to provide you with any additional data ... His research is published in the peer-reviewed literature which has passed muster with the editors of those journals and other scientists who have reviewed his manuscripts. You are free to your analysis of climate data and he is free to his. Subsequently, a senior NSF official said that dissemination of data was merely up to the "professional judgement" of the researchers. Ironically, the NAS panel relied heavily on unarchived data. In general, we allow researchers the freedom to convey their scientific results in a manner consistent with their professional judgement... The Department of Energy funded the development of the well-known CRU instrumental temperature series, used by IPCC and others. In response to a request for supporting data, Philip Jones, a prominent researcher said: We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it? Although DOE had funded the collection, their past and present grant agreements had not ensured that even DOE had access to the supporting data and they said that they were unable to assist. Phil [is] not obligated under the conditions of past or present DOE proposal awards to provide these items to CDIAC. I regret we cannot furnish the materials you seek In conclusion, I re-iterate that you can place little reliance on any existing multiproxy study; that you need to distinguish between the limited due diligence of journal peer review and the substantive due diligence of an audit; and that simple administrative measures can substantially improve paleoclimate research practices. Both the NAS report and Wegman reports are valuable studies by accomplished authors. Nothing that I say here should be construed as diminishing the seriousness of climate change as a public issue. It is precisely because it is a serious issue that policy-makers are entitled to the best possible information and should ensure that data, code and methods be accurately and completely archived and discourage practices that interfere with scientific reproducibility. References: See NAS Panel report. Presentation to the Subcommittee on Oversight and Investigations of the House Energy and Commerce Committee. Stephen McIntyre Toronto Ontario Washington DC, July 19, 2006. 1 Overview: 1. little reliance can be placed on the original MBHreconstruction, various efforts to salvage it or onother similar studies, even ones which do not useMann's principal components methodology; 2. peer review as practiced by academic journals isnot an audit, but something much more limited. Literature review is not independent due diligence. 3. much of the work in dispute is funded by the U.S. federal government. Some very simpleadministrative measur