3.5.2 Evaluating model performance

Section 3.5.1 stressed the absolute necessity of testing the quality of the model results fully. Here we will present some of the standard simulations that could be performed. However, we will not discuss the tests specifically designed to analyse the accuracy of numerical methods or of a particular parameterisation.

The first requirement is that the model is able to simulate reasonably well the climate in recent decades for which we have good estimates (Fig. 3.16). This implies performing simulations including the evolution of both natural and anthropogenic forcings (section 5.5.2) over that period. Numerical experiments with a constant forcing set at the mean for recent decades or for pre-industrial conditions (i.e. before any significant anthropogenic forcing, generally 1750 or 1850) can also be conducted in order to characterise a quasi-equilibrium behaviour of the model. In this case, it is necessary to take into account the difference between the pre-industrial conditions simulated by the model and present-day observations.

In these simulations, the long-term average of various variables, in all the model components, is compared with observations, generally interpolated on a common grid. Furthermore, the ability of the model to reproduce the observed climate variability on all time scales must be checked. This ranges from the relatively high frequency variations characteristic of temperature extremes such as heat waves to the most important modes of large-scale variability such as the El Niño-Southern Oscillation and the North Atlantic Oscillation (see section 5.2). Finally when driven by an adequate forcing, the climate models must be able to reproduce the observed warming of the Earth’s surface over the last 150 years as well as the other recent climate changes.

Recent decades only cover a small fraction of the climate variations observed since the Earth’s formation (see Chapter 5) and expected in the future (see Chapter 6). To test the ability of models to reproduce different climates, it is thus necessary to try to simulate some past conditions. The quality of the available observational data is (much) lower than for recent decades and it may sometimes be hard to draw reliable conclusions from model/data comparisons for some past periods. Nevertheless, that is the only sample of possible states of the climate system that is available to us.

The second natural test period (Fig. 3.16) is the Holocene and the last millennium, for which we have a reasonably good knowledge of climate variations (see section 5.5). Although significant uncertainties are present, the forcing is much better known than for earlier periods. Furthermore, the boundary conditions (such as the topography or ocean bathymetry, see section 1.5) are similar to the present ones. The last glacial maximum is also a key period because it represents a relatively recent climate clearly different from that of recent decades (see section 5.4.2). In order to perform such simulations, unless the variables are computed interactively, it is necessary to specify variables such as the position and shape of the large ice sheets present on continents, the changes in the land/sea boundaries and ocean depth due to the lower sea levels, the modification in the vegetation cover and in the radiative properties of the atmosphere (in particular due to the higher dust content). All these elements can be sources of uncertainty for the climate simulation. Pre-quaternary climates (see section 5.3) offer an even wider range of climate variations but the uncertainties on the forcing, boundary conditions and the climate itself are larger. As a consequence, these periods are not currently used as standard tests for climate models, although this will probably change in the near future as new information becomes available.

Finally some idealised experiments are performed with climate models (Fig. 3.16). These could not be directly compared to observations as they do not correspond to any past or present situation. However, they are very useful to document the model response to a simple, well-defined perturbation. Two standard thought-experiments are generally conducted. The first is a doubling of the atmospheric CO₂ concentration in the model, a test required to estimate climate sensitivity of the model (see section 4.1.3). In the second (water hosing), large amounts of freshwater are poured into the North Atlantic to analyse the climate changes induced by the associated modification of the oceanic circulation (see section 5.5.1). These tests also allow the behaviour of different models to be compared in exactly the same experimental conditions. This leads to model inter-comparison exercises whose goals are a better understanding of the causes of the different responses of the various models. The results of such inter-comparisons are archived in data bases to ensure wide access. The results of other simulations (for example, mid-Holocene or last glacial maximum climates, climate change during recent decades, future climate change) are also stored in public or semi-public databases so that they can be analysed independently by large numbers of scientists.

---