A mode-based metric for evaluating global climate models

Kent, Michael L

A mode-based metric for evaluating global climate models

Doctoral Thesis

2018

Publisher

University of Cape Town

Department

Department of Environmental and Geographical Science

Faculty

Faculty of Science

Abstract

Climate models are software tools that simulate the climate system and require evaluation to assess their skill, guide their development, and assist in selecting model simulations from among the many different ones available. There are a variety of methods and approaches that can be used to evaluate models. But there is no one best method and many possible and valid approaches exist. Models contain inherent uncertainties which complicate their evaluation, and include limitations in the knowledge of climate process dynamics and structural errors in constructing the models. Similar to the multiplicity of methods for the evaluation of model simulations, there also exist many possible approaches to addressing these sources of uncertainty. The challenge with uncertainty, is the difficulty in disaggregating it from the underlying element of legitimate chaotic behaviour in complex systems. In response, this dissertation is primarily one of methodological development to contribute to new ways of addressing the model evaluation challenge. The work defines and demonstrates a new evaluation method which complements the existing toolset. Specifically, the method defines a model performance metric that focuses on the extent to which a model is able to simulate global modes of climate variability (modes, e.g.: ENSO) evident in the observed climate data. Modes are one aspect of the climate that can be evaluated and are fundamental to model skill. Therefore their credible simulation is a necessary (but not sufficient) condition to ensuring that models are producing the right result (appropriate variability on the range of spatial and temporal scales) for the right reason. By ranking models by this metric of their skill in capturing fundamental global modes, poorly performing model simulations can be identified for potential exclusion (discounted). This metric therefore serves as a potential method to assist in the management of uncertainty when assessing multi-model data. The method develops a novel application of Independent Component Analysis (ICA). ICA is used to find representations of modes in a record of the present day climate (represented by reanalysis data), and then their degree of manifestation in global models is assessed. Recognising the large volume of model data (highly autocorrelated in space and time) the technique includes a data reduction technique to facilitate the evaluation of multiple model simulations. The technique also includes a novel measure of variance to differentiate it from a similar technique (Principal Component Analysis), and offers an approach to improve the consistency of results (signals) when using an unmixing matrix initialized with random values. As reanalysis data is itself a model product (constrained by observations), the performance metric is tested for its strength in discriminating modes by using two different reanalysis datasets and a dataset containing only Gaussian noise. The metric is found to perform predictably, and clearly demonstrates the ability to discriminate signal from noise when using geopotential height (GHT, 700mb and 500mb) and near surface air temperature data (TAS). The dependency of model performance on the variable measured by any metric can be a problem for model evaluation, as it introduces the choice of which variable should be measured to assess model performance. The ICA-based metric is found to be slightly less sensitive to a change in model rank between GHT (700mb) and TAS, compared to a similar novel variance metric (Fourier Distance) and a mean climate metric (bias). The ICA application is also found to produce plausible representations of modes (static maps), while a direct association to known modes is left for future work due to inherit complexities. The plausibility, consistency, and rank sensitivity of the novel application of ICA, suggests it has value in assisting the evaluation of multi-model datasets and the ensemble members for any one model.

Keywords

Climate models

Reference:

Collections

PhD / Doctoral

Full item page