En:metrics

De WAVEWATCH-III Wiki

Metrics

By definition a “metric” is a measure of the model performance, but it does not have to be a number.

Different kinds of metrics

First of all why do we need metrics?

  • to understand and quantify the model behaviour
  • to measure a model performance, which can be used to verify that the model performs as expected or not

Metrics are built from modelled variables, which in the case of phase-averaged wave models must all be defined from the wave spectrum. These variables can be either measured more or less directly,

  • significant wave height
  • mean periods
  • directional spreads
  • mean square slopes

or be a insightful diagnostic of the model behaviour:

  • speed of rotation of the mean wave direction
  • wave-supported stress

(these two can also be measured in a way … not a very good example)

From these variables one can do several levels of metrics (see the GODAE ocean circulation metrics as examples):

  • simple statistics: bias, normalized bias, normalized RMS error, scatter index, correlation coefficient. These are derived from a set of model values X_m compared to a set of observed values X_o
  • patterns: sensitivity of variable X to other variables Y and Z … : for example the mean square slope changes with wind speed and wave height, and the pattern of this change is very useful for calibrating the cumulative dissipation effect.