Homogenization seminar

People interested in this blog are likely also interested in the upcoming
Seventh seminar for homogenization and quality control in climatological databases and COST ES-0601 “HOME” action management committee and final meeting in Budapest, Hungary on the 24 – 28 October 2011. Abstract deadline is the 16th of
September 2011. This is the main event in the homogenization community, in my opinion.

If I had but one analog I could create ...

I think I would look to create something that tested in some sense the limits of homogenization methods whilst still retaining some realism.

I would take a forced component run such as c20c and then look to add change points that in the net removed that trend. This would penalize any algorithm that tended to introduce adjustments with a preferential zero bias.

The breaks I would add would be a mix of step like and slope like and a large number would have changes in seasonality and timeseries variance associated.

I'd have limited metadata and what metadata there was would be poor quality.

I would assume that most breaks were small (sigma <1K, in some cases perhaps <<1K) and that they happened fairly frequently (once every 5 to ten years say on average).

A number of breaks would be quasi-contemperaneous over countries and these would have very similar characteristics to each other.

There is documented evidence that these issues all to some extent pervade the network (e.g. US network move from stevenson screen to automated sensors happened largely within 5 years over 70% of the network).

So, whilst at the outer bounds of plausibility it would not be an entirely implausible error structure.

Big questions with which to test homogenisation algorithms

Hi, I'm hoping to get time in the call to touch on this a little. Similar to previous posts about worse nightmares I would like us to think about questions that we want to answer with the analog-error-models. The idea would be to pick 8 of these to run with for each 3 year benchmarking cycle - one for each world. If these start from something simple to something horrible this will give us a chance to see where algorithms begin to struggle. Lets focus here on monthly means to make things a little simpler for the time being - please comment with your ideas. For example:

1) Do homogenisation algorithms detect discontinuities when none are present?
Analog-error-world 1 = A historical forcing model analog-known-world with no-errors added

2) Do homogenisation algorithms cope with discontinuities that affect the variance?
Analog-error-world 2 = A historical forcing model analog-known-world with seasonally constant changes applied
Analog-error-world 3 = A historical forcing model analog-known-world with seasonally varying changes applied at the same location and approximate magnitude as analog-error-world 2

2) Can homogenisation algorithms cope with non-stationary worlds/ where there is a background trend?
Analog-error-world 4 = A control forcing (constant pre-industrial emissions) model analog-known-world with mixed error structure applied
Analog-error-world 5 = An A1B (high emissions) forcing model analog-known-world basis with identical error structure to World 4

3) Can homogenisation algorithms cope when discontinuities are small and frequent?
Analog-error-world 6 = A historical forcing model analog-known-world with many small discontinuities added of various sign biases - (seasonally varying to be realistic?).

4) Can homogenisation algorithms cope with layered gradual and abrupt discontinuities (i.e., urban warming + instrument shelter change)
Analog-error-world 7 = A historical forcing model analog-known-world with either gradual or abrupt discontinuities applied to a station (seasonally varying to be realistic?)
Analog-error-world 8 = A historical forcing model analog-known-world with both gradual and abrupt discontinuities applied to a station (seasonally varying to be realistic?) using the initial error structure from analog-error-world 7 with other errors added.

At present its probably useful just to come up with as many plausible questions as possible and examples of error world structures to explore these.

There is an argument for including a really nasty one that is perhaps unplausible - so feel free to be creative.

Creating the Benchmark 'Truths'

The Steering Committee is drafting an Implementation Plan to ensure success of the Surface Temperature Initiative. This will be posted on the website (www.surfacetemperatures.org) when finalised. Crucially it has a list of deadlines - some of which relate to our Benchmarking and Assessment Working group. These are our goals:

1. Defining methods to create the benchmark analog truth stations - to mirror the databank consolidated master database (these will not be made publicly available immediately)

2. Defining the spread of error models to be applied to the benchmark analog truths

3. Creating the benchmark analog truths and error worlds - these will be publicly available

4. Running some kind of review workshop (possibly online) at the end of the 3 year cycle to release the 'truths' and review benchmark production/implementation - can this be improved for the second cycle.

I would like to try and iron out goal 1. in the coming conference call (Wed March 30th 2pm GMT) GMT). I think we have two options - purely synthetic utilising statistical models or part synthetic using a combination of physical models (GCMs or reanalyses) and statistical models. Both will likely need to use information from the databank consolidated master database to govern individual station climate characteristics.

I think it is important to characterise true climate features of real data for each station - so its climatology, variance, background trend, natural variability, serial autocorrelation and its relationship to other stations within the spatial covariance structure.

Pure Synthetic

X(t,l,h) = S(t,l,h) + T(t,l,h) + RE(t,l,h)

Where X = station at time t, location l and height h, S = seasonal cycles, T = background features (e.g., trend, ENSO, volcanoes, solar cycles etc.) and RE = residual random error.

Could the actual stations be used to give basic climate characteristics? I'm not too worried about mimicking stations exactly but we do want to represent the spatial covariance between a global network of stations quite accurately (Tropics, mid-latitudes, coastal, mountainous, etc.) and the station 'noise' around the errors that we eventually apply which will be our signal.

Part Synthetic

This would take gridded fields of GCM or 20th Century Reanalyses data (no inhomogeneities due to station moves, instrument changes or data type ingestion changes) as the base for creating analog stations to mimic the consolidated master database. The grids would have to be downscaled, using statistical models and actual stations from the database to give the basic climate characteristics - similar to the above. Here realistic S and T are already there in the models - they just need tweaking to create individual stations with appropriate autocorrelation and within a realistic spatial covariance structure. RE would need to be added.

So there are some ideas to get us started and bash to pieces. I'm not a statistician, and so certainly need help!

Assessing the Benchmarks

When we eventually design the benchmarks we need to keep in mind our assessment aims and outputs. They need to be designed such that it is easy to obtain useful and meaningful measures of skill. I've been chatting to Ian Jolliffe about the use of simple forecast verification techniques (break present and break detected = very good, break present and break not detected = bad, no break and break detected = very bad, no break and no break detected = good) including ROC plots. I'll go into more detail on this at some point soon but just thought I would throw the thought out there in case anyone has any bright ideas/thoughts.

Review paper references

Hi All, all the comments so far have given me a lot to think about which is great. The references are really useful. I'm hoping to get chance to start the review paper this weekend so if anyone can think of anymore useful references for me to browse please can you list them here.

My first time using blog...

I had a look at a number of homogeneous stations (stations with no major problems) to retrieve some statistical properties. I used the monthly means of the daily maximum temperature for about 30 stations, covering as much as possible 1920-2009. I examined the monthly departures from their long-term means (12 monthly means) so each series had about 1080 data points. Overall, the best-fit linear trend ranged from 0.3 to 1.5°C for 1920-2009. Once the trend was removed, the standard deviation ranged from 2.5 to 3.5 and the autocorrelation from 0.15 to 0.25. The reverse process can be used to generate datasets. However, this approach does not capture the long-term variation, such as warming from the 1900s to the 1930s, followed by a cooling to the 1970s, and warming to the 2000s.

The types of “inhomogeneities” that I have seen in temperature series are:

1. change in annual means: 2 or 3 steps are common over 90 years; a more complex situation would be a step every 10 years
2. change in monthly means: here, the magnitude of the step would be different depending of the month (e.g. an instrument relocation could generate a positive step in the summer and a negative step in the winter)
3. change in mean and variance due to a change of instrument or observing practices
4. change in mean followed by a change in trend direction: this occurs rarely and I can’t think of a physical process that would generate this situation in temperature series.

These “inhomogeneities” could be detected using a network of neighbour stations which could be highly (or not) correlated with the tested series. In addition, the same (or other) “inhomogeneities” could be found in the neighbouring series.