• No results found

Bias Correction & Forecast Skill

N/A
N/A
Protected

Academic year: 2022

Share "Bias Correction & Forecast Skill"

Copied!
47
0
0

Bezig met laden.... (Bekijk nu de volledige tekst)

Hele tekst

(1)

1

Climate Test Bed Seminar Series 24 June 2009

Bias Correction & Forecast Skill

of NCEP GFS Ensemble Week 1 & Week 2 Precipitation & Soil Moisture Forecasts

Yun Fan & Huug van den Dool

Acknowledgement: Jae Schemm, John Janowiak, Doug Lecomte, Jin Huang, Pingping Xie, Viviane Silva, Peitao Peng, Vern Kousky, Wayne Higgins

(2)

2

Outline

• Motivation

• Methodology

• Performance of NCEP GFS Week1 & Week2 Ensemble Precipitation Forecasts

• Analysis of Week1 & Week2 Biases & Errors

• Application: land model forced with bias corrected week1 week2 P & T2m forecast

• Future Work

(3)

3

History of Soil Moisture “Dynamical” Outlook CPC Leaky Bucket Hydrological Model

Forced With Week 1 & Week 2 GFS Forecasts

Single member HR MRF (started around 1997 & CONUS) Ensemble GFS (started late 2001 & CONUS)

Bias corrected Ensemble GFS (started late 2003 & CONUS) Bias corrected Ensemble GFS (started late 2007 & global land)

:

The prediction skill of soil moisture crucially depends on our

ability to predict precipitation

Early stage

(

both good and bad comments

)

Recent years

(

more & more good comments

)

So its time to verify & quantify:

daily GFS ensemble week 1 & week 2 precip forecast skills & statistics

(4)

4

The quality of soil moisture prediction largely or almost

entirely depends on the quality of precipitation prediction

(5)

5

Daily bias correction based on last 30 (or 7) day forecast errors

Week1

Future Week2

Today

Past

Last 30 day

1/N Σ [ Pf (week1) – Po (week1) ] = Bias1 1/N Σ [ Pf (week2) – Po (week2) ] = Bias2

Pf : GFS ensemble week1 & week2 precip forecast

Po: Observed week1 & week2 precip from CPC daily global Unified Precip N = ( 30, 7..….)

(6)

6

North America

Seasonal cycle with

Large day to day fluctuation

On 0.5x0.5 obs grid

(7)

7

South America

On 0.5x0.5 obs grid

(8)

8

Asia-Australia

On 0.5x0.5 obs grid

(9)

9

Africa

On 0.5x0.5 obs grid

(10)

10

How good is GFS?

On 0.5x0.5 obs grid Seasonal cycle with Large day to day fluctuation

(11)

11

On 0.5x0.5 obs grid

How good is GFS?

Seasonal cycle with Large day to day fluctuation

(12)

12

Comparison

(based on last 30-day forecast errors)

Obs grids (regrid model grids to 0.5x0.5 obs grids)

Model grids (regrid obs grids to 2.5x2.5 model grids)

Question: Does grid matter for skills assessment ?

(13)

Skill does not depend 13

much on the grid

(14)

14

Comparison

bias corrected skills

(based on last 30-day forecast errors)

bias corrected skills

(based on last 7-day forecast errors)

Question: Does the bias estimate influence skill ?

Week1

Future Week2

Today

Past

Last 30 day

1/N Σ [ Pf (week1) – Po (week1) ] = Bias1 1/N Σ [ Pf (week2) – Po (week2) ] = Bias2

Pf : GFS ensemble week1 & week2 precip forecast

Po: Observed week1 & week2 precip from CPC daily global Unified Precip

N = ( 30, 7..….)

(15)

1) Skill depends on the definition of bias 15

2) 30-day bias correction better than 7-day bias correction

(16)

16

Comparison

bias corrected skills

(based on last 30-day forecast errors)

raw forecast skills

(no bias correction applied)

Question: Does bias correction improve skill

in terms of Spatial Correlation and RMSE ?

(17)

17

Bias correction is time &

location dependent

(18)

18

Bias correction helps everywhere

(19)

19 Table 1. Averaged (May 1, 2008 – June 7, 2009) spatial

correlations over different monsoon regions

Week 1 Week 2

Bias

Correction No Bias

Correction Bias

Correction No Bias Correction North

America 0.49 0.48 0.24 0.26

South

America 0.45 0.25 0.31 0.18

Asia

Australia 0.47 0.40 0.29 0.26

Africa 0.40 0.24 0.25 0.13

Table 2. Averaged (May 1, 2008 – June 7, 2009) RMSE over different monsoon regions (unit: mm/week)

Week 1 Week 2

Bias

Correction No Bias

Correction Bias

Correction No Bias Correction North

America 19.18 22.82 21.61 23.58

South

America 29.55 41.06 32.27 41.72

Asia

Australia 22.65 27.62 25.24 29.15

Africa 17.06 19.47 17.66 19.33

The effectiveness of bias correction is mainly space dependent.

Bias correction can correct spatial distribution of Pf & reduce its error.

Similarity of Pf & Po

Distance of Pf & Po

Reduced by 28%

Reduced by 23%

Increased by 80%

Increased by 67%

(20)

20

CONUS

30-day running mean

(21)

21

In terms of Spatial Anomaly Correlation,

bias correction helps :

1) very little over North America

2) considerably over South America & Africa 3) a little over Asia-Australia

In terms of RMSE:

Bias correction helps everywhere

Questions:

Why bias correction works but varies in space and time?

What biases look like?

Are biases removable & to what extent are they removable?

(22)

22

Temporal-spatial structures of last 30-day biases:

Daily Bias1 & Bias2 used to correct GFS ensemble week1 & week2 forecasts

Week1

Future Week2

Today

Past

Last 30 day

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1 1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2

M

m

m m

t EOF s PC

Mean t

s Bias

1

) ( )

( )

,

(

2

,

1

(23)

23

Annual Mean Bias or Raw Forecast Error

Week-1 mean Bias

Week-2 mean Bias

(24)

24

Mean Bias of Daily R2 & Observed Precip (1979-2006)

(25)

25

summer

winter

(26)

26

winter summer

(27)

27

Temporal-spatial structures of last 30-day biases:

Daily Bias1 & Bias2 used to correct GFS ensemble week1 & week2 forecasts

Week1

Future Week2

Today

Past

Last 30 day

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1 1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2

Large-scale & low-frequency (annual or semi-annual cycles) are prominent

First two EOF modes of Bias1 & Bias2 explain about 60% total variances

GFS has prominent annual cycle errors

(lesson for model development?)

(28)

28

Temporal-spatial structures of real time raw forecast errors:

Daily GFS week1 & week2 forecast errors

without bias correction

Week1

Future Week2

Today

Past

Last 30 day

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1 1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2

Pf (week1) – Po (week1) = Error1 Pf (week2) – Po (week2) = Error2

No bias correction applied

M

m

m m

t EOF s PC

Mean t

s Error

1

) ( )

( )

,

(

2

,

1

(29)

29

(30)

30

(31)

31

Temporal-spatial structures of real time raw forecast errors:

Daily GFS week1 & week2 forecast errors without bias correction

Week1

Future Week2

Today

Past

Last 30 day

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1 1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2 Pf (week1) – Po (week1) = Error1 Pf (week2) – Po (week2) = Error2

No bias correction applied

Raw forecast errors are dominated by the 1

st,

2

nd

or 3

rd

EOFs in Bias1 & Bias2

First two EOF modes of Error1 & Error2 explain about 23~35% total variances

At least this amount of error is removable. But so far bias correction was not done by EOF analysis

(32)

32

Temporal-spatial structures of real time forecast errors:

GFS week1 & week2 forecast errors

with last 30-day bias correction

Week1

Future Week2

Today

Past

Last 30 day

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1 1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2 Pf (week1) – Po (week1) = Error1 Pf (week2) – Po (week2) = Error2

Bias correction:

Error1 = Error1 – Bias1 Error2 = Error2 – Bias2

M

m

m m

t EOF s PC

Mean t

s Error

1

) ( )

( )

,

(

2

,

1

(33)

33

Annual Mean Forecast Error after bias correction

5 times smaller than mean bias or raw forecast error

Week-1 mean forecast error

Week-2 mean forecast error

(34)

34

(35)

35

(36)

36

Temporal-spatial structures of real time forecast errors:

GFS week1 & week2 forecast errors with last 30-day bias correction

Week1

Future Week2

Today

Past

Last 30 day

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1 1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2 Pf (week1) – Po (week1) = Error1 Pf (week2) – Po (week2) = Error2

Bias correction:

Error1 = Error1 – Bias1 Error2 = Error2 – Bias2

Bias Corrected Forecast Errors are much more random (in time mainly, EOFs more “white”).

Leading EOF modes of Bias1, Bias2, & Error1, Error2 Show that GFS has prominent large-scale & low-

frequency errors or GFS has difficulty to reproduce those observed Precip patterns & their evolution.

However, to some extent they can be corrected

through bias correction, especially in winter season.

(37)

37

Application

Soil Moisture “Dynamical” Outlook

CPC Leaky Bucket Hydrological Model

Forced With Week-1 & Week-2 GFS Ensemble Forecasts

(Daily data from 01Nov2003 to present)

All initial conditions & verification datasets are from leaky

bucket model forced with daily observed P & T2m

(38)

38

Some Thoughts:

• Once this (SST, w) was the lower boundary….

• Both SST and w have (high) persistence

• Old ‘standard’ in meteorology: If you cannot beat persistence …..

• For instance: dw/dt = P – E - R = F or w(t+1)=w(t) + F

• Clearly if we do not know F with sufficient skill, the

forecast loses against persistence (F=0).

(39)

39

(40)

40

(41)

41

(42)

42 30-day running mean

P1=0.9511, C1=0.9512 PR1=16.27,

FR1=18.02

P2=0.9015, C2=0.8957 PR2=23.67,

FR2=26.56

(43)

30-day running mean 43 Precip

(44)

44

Even moderate forecast skill at right time still help a lot

(45)

45 30-day running mean for week-2

Hybrid persistence = week-1 forecast persists to week-2

(46)

46

• Moderate week-1 & week-2 GFS P forecast skills

• Last 30-d biases dominated by low-frequency &

large-scale errors

• Bias corrections are time & location dependent

• Soil moisture forecast skill hardly beats its persistence over CONUS

• The inability to outperform persistence relates to the skill of precipitation not being above a threshold (AC>0.5 is required)

Summary

(47)

47

• Is PDF bias correction better?

• GFS Week3 & Week4 Precip Assessment

• GFS hindcasts?

• How about New CFSRR?

Future Work

Referenties

GERELATEERDE DOCUMENTEN

In hydrological application in the Zambezi Basin, previous studies report on the use of uncorrected SREs despite evidence of errors in satellite products (Cohen et al., 2012).

Following the literature on teacher bias, the analyses rely on differences between subjective teacher evaluations (track recommendations) and objective measures of a child’s

Instead of attempts to curb commercialisation of the Colleges, which may result in their extinction, what is needed is a comprehensive examination of the contribution private

Aan die hand van die literatuurstudie is 'n vraelys ontwikkel waarin persepsies van intrinsieke bevorderingshindernisse wat by onderwyseresse bestaan, getoets is.

F3 F3 158-206 158-206 Severe – Roofs and some walls torn off Severe – Roofs and some walls torn off well constructed homes, trains.. well constructed homes, trains

Huehuetenango and San Marcos in Guatemala caused landslides, flooding, and strong ocean surges along the southern Pacific coast. Those conditions persisted during the

terugkoppeling naar het medewerkerniveau niet goed verloopt.. Op medewerkerniveau is er vrijwel geen cross-functionele communicatie in het financiële forecasting proces. Indien

1) A price is much higher than the normal price, which is called an “abnormal high price”. 2) The difference between two neighboring prices is larger than a