11 November 2009

Response to Gavin Schmidt on Klotzbach et al. 2009

UPDATE # 11/8/11: Steve McIntyre, aided by some new information from Gavin, nicely resolves the dispute over amplifications factors and finds Gavin to be in error and agrees with the calculations in Klotzbach et al. 2010.  The Figure above comes from McIntyre's nice illustrative example. 

An update over at Real Climate would be appropriate -- "Science would work a lot faster if authors corrected their own work when errors were found.” -- Gavin Schmidt on Klotzbach et al.  Here is where we corrected our errors -- Gavin, how about you?

UPDATE 2: For an example of how Gavin uses of his comment threads (and why I see little use commenting there) see my submission and Gavin's edits/response. Gavin's reaction helps to explain why such disputes must take place in the peer reviewed literature rather than on blogs, and I hope that Gavin chooses to follow up with a submission.

UPDATE: A predicted below, Gavin uses the comment thread at Real Climate to raise (red herring) issues that he did not feel the need to address in his main post. My father
deals with Gavin's supplementary concerns.

Over at Real Climate, Gavin Schmidt is upset that we did not properly reference the GISS model as a source for a citation in Klotzbach et al. (2009, PDF). He is right to be upset as we failed to make that addition during page proofs. We have submitted a corrigendum as follows:
"A reference in Klotzbach et al. (2009) to McKitrick (pers. corres.) depended upon an analysis of GISS model output placed online by Gavin Schmidt of NASA in early 2009. We regret the oversight."
But Gavin goes too far in his critical comments when he states that we have made an "error" because we did not change aspects of our paper based on informal comments that he provided us in August based on a quick and dirty recalculation he did. For Gavin, not doing what he demands means that we have made an "error" and in typical overheated fashion -- "muddied the peer reviewed literature." His comments are surely going to provoke another round of mud slinging and misinformation about our work, and I rather suspect that this is the point, as in the Real Climate post Gavin encourages his commenters to have a go at us on issues unrelated to his post in the comment thread. Uncool, but not unexpected. Make no mistake, climate science is a contact sport.

We have repeatedly pointed out to Gavin that our results are insensitive to the points that he raises, and we showed this to him based on a recalculation of our analysis using his numbers when he first raised the issue. So were we to have changed our paper it would have said something like:
Informally, Gavin Schmidt raised some interesting points that are worthy of further investigation, however they make no difference to our conclusions. They do however hint at broader problems in reconciling surface and tropospheric tempertures that go well beyond the scope of our paper.
It looks like he completely understands the full insignificance of his comments as he writes at Real Climate that this situation is about "when problems (of whatever seriousness) are pointed out." You can be sure that he put in "of whatever seriousness" because in this instance his complaint is "not very serious." In any case, if Gavin thinks that the issue is important enough, we encourage him to submit something to the peer reviewed literature and explain why this matters. Given that his complaint does not materially affect our results and opens up a Pandora's Box of trouble for the models, I really doubt that he will do so. We also invited him on several occasions to join us in preparing a follow-on paper looking more broadly at the amplification issue, and he declined that invitation. Of course, had he taken us up on our offer (which still stands), he would have not been able to write a misleading post at Real Climate about (ironically, enough) muddying the waters.

Gavin kindly shared an early version blog post in advance of publishing. Below is how the authors of Klotzbach et al. 2009 responded. (Note: In the past Gavin has requested that we do not post his emailed comments, so we have not. He is free to add these emails in the comments if he'd like.) We look forward to further discussing this issue with Gavin in the peer reviewed literature.

Meantime, the bottom line is that regardless of the invective that might be found over in the Real Climate comment thread and among those who are inevitably going to (mis)citing that post, Gavin's comments do absolutely noting to impeach the conclusions of Klotzbach et al. (2009).

Here is the reply we sent earlier today:
Dear Gavin-

Thanks for sending the draft blog post. Below we identify several errors and omissions in it, and we ask you to correct these before publishing to avoid misrepresenting our paper and its conclusions.

But first, let us apologize for failing to add the reference to the supplementary information from Schmidt (2009) as the source of Ross McKitrick's calculations. This referencing oversight was simply overlooked at the proof stage, so thanks for calling it to our attention. We will submit a Corrigendum to JGR-A to rectify this. We also note that you replaced the data McKitrick relied on in our pers. corres. citation with a new dataset on 16 August 2009, which means that the data relied on by McKitrick were replaced on the very same day that you were raising these comments with us. Now that the original data has been removed, that certainly muddies the waters a bit as far as correctly citing the information.

There are also some factual issues with your blog post.

First, you strongly suggest that our paper has "fundamental flaws" but never say what these are. It is misleading at best to intimate that there are "fundamental flaws" when the issues you raise are irrelevant to the analysis in the paper. We'd ask that you specifically identify what you see as "fundamental flaws" if you choose to use such language. Otherwise your post is highly misleading.

Second, we published an exchange with you online in which we showed that our results were insensitive to the amplification provided by McKitrick versus those that you provided. Given that there was no implication to our conclusions, the issue that you raise, while certainly worth further investigation across models, is not central to our paper. At a minimum your blog post should clearly state that we have shown that our statistical results and conclusions are not influenced by this issue, and to be fair you should link to and present our demonstration of this, which you allude to but do not share in the post.

When you write, "a key assumption for this study rest solely on a personal communication from an economics professor" you are simply incorrect. The assumption, as we have shown to you, is not "key." Again, the issue that you raise is tangential to the paper and does not affect its conclusions.

As we understand the information provided to us by McKitrick, he calculated the model ratio based on model grids for which there were also observed data (so he could do an apples to apples comparison with observations and models). We suspect that you are using all land grids from the model. Thus, when we used observations for the surface temperature, we were being more consistent by using the subset of grids Ross developed as compared to those that you presented. At the same time, clearly our paper was not meant to be the final word on model amplification (nor of course was your note to us), since we show only a single illustrative result, and given the wide range of amplifications across the various models this is certainly a subject worth further study. But again, it is not directly relevant to our analysis.

Third, you are incorrect when you write that the choice of amplification factor "leaves one of the key hypotheses of the Klotzbach paper somewhat devoid of empirical support." Again, as we have shown, the conclusions of the paper are robust to choice of amplification factor. We would agree that a closer look at amplification factors over land and ocean as compared with the observational record is worthy of attention, which is why we invited you to co-author a follow-up piece to more fully explore this topic, and to which you declined to participate. The information that you provide suggests that there are even deeper problems in reconciling models with observations, but that subject goes well beyond the scope of our paper. But let’s be very clear, our results are not affected by the issues that you raise.

In sum, we are sorry to have overlooked the reference to the NASA GISS model as the primary source of a personal correspondence reference in the paper, and understand why you might feel aggrieved by this. We will submit a corrigendum JGR-A to present the acknowledgment more fully. We apologize. At the same time we do find it odd that you removed the data that McKitrick relied on from the GISS website and replaced it on the same day that you provided comments to us in August.

Finally, even though we understand why you may be upset about the reference, we see this unfortunate oversight on our part as providing no justification for you to mislead people about our paper or its findings, and we'd respectfully ask that you present information correctly and fully. If you feel that you have substantial scientific disagreements with our paper, we would encourage you to either take up our offer of collaboration on a subsequent piece, or to pursue this through the peer-reviewed literature on your own. As you have correctly emphasized in the past, blog commentary is not a substitute for the peer reviewed literature.

Sincerely,

Phil, John, Dick, Roger and Roger