Research to Operations (R2O) and S2S forecast and verification products development

Last modified by s2s_wiki on 2020/09/12 09:43

Highlights: The WWRP/WGNE Joint Working Group on Forecast Verification Research (JWGFVR) is pleased to announce the around-the-clock 2020 International Verification Methods Workshop Online (2020-IVMW-O), which will be held virtually, during two weeks, from the 9th to the 13th and from the 16th to the 20th of November 2020. This workshop will consist in 2-hour online daily sessions, with live-stream presentations and discussion.

The S2S community is cordially invited to submit abstracts on S2S verification methodologies to the workshop. Registration and abstract submission is open at the workshop website https://jwgfvr.univie.ac.at/

Please note the following timeline:

Abstract submission opens: 30 June 2020
Registration opens: 30 June 2020
Abstract submission closed: 7 September 2020
Registration closes: 30 September 2020
Notification of Acceptance: 30 September 2020
Presentation submission: 30 October 2020
Online Workshop: 9-13 and 16-20 November 2020

The WWRP/WGNE Joint Working Group on Forecast Verification Research (JWGFVR) is also conducting the 2^nd International Verification Challenge – Seeking the Best New Verification Metrics Making Use of Non-Traditional Observations. The contest is in support of the WWRP projects on High Impact Weather, Subseasonal to Seasonal Prediction (S2S), and Polar Prediction (PPP). All interested researchers and practitioners are warmly encouraged to participate.

The deadline for entries is 30 April 2021. For additional information, including the entry form to participate, please visit the 2nd International Verification Challenge website or contact verifchallenge@bom.gov.au.

Membership of Research to Operations (R2O) and S2S forecast and verification products development sub-project

Caio Coelho (CPTEC/INPE, Brazil)
Andrew Robertson (IRI, USA)
Arun Kumar (NOAA, USA)
Yuhei Takaya (JMA, Japan)
Anca Brookshaw (ECMWF)
Debra Hudson (BoM, Australia)
Angel Muñoz (IRI, USA)
Joanne Robbins (UKMO, UK)

1) Scientific and Operational Objectives

Pursue research for testing and developing methodologies for calibration, multi-model combination, verification and generation of forecast products.
Coordinate with the relevant WMO technical commissions to define the standards and protocols for operational implementation and exchange of S2S forecasts such that by the end of the Phase II of the S2S, the infrastructure related to the data exchange to support research can be transitioned into the operational domain.

The R2O and S2S forecast and verification products activities plan provides additional information about this sub-project, which builds on the work of the previous (Phase I) S2S Verification and Products sub-project.

2) Linkages with WMO activities

On the research side this sub-project has linkages with the Joint Working Group on Forecast Verification Research (JWGFVR), a WMO joint working group of the Working Group on Numerical Experimentation (WGNE) and the World Weather Research Programme (WWRP). This sub-project also has synergies with the World Climate Research Programme (WCRP) Working Group on Subseasonal to Interdecadal Prediction (WGSIP) through the Climate forecast information for decision making (I4D) project.

On the operational side, this sub-project has linkages with Inter-Programme Expert Team on Operational Predictions from Sub-seasonal to Longer-Time Scale (IPET-OPSLS), a WMO joint team of the Commission for Basic Systems (CBS) and the Commission for Climatology (CCl).

3) Proposed questions to be addressed

The World Weather Research Programme (WWRP) has flagged improving forecasts of precipitation over land as an important area for S2S to focus research and services development efforts. In order to help advance scientific knowledge and the development of forecast and verification products in this priority area this sub-project invites the S2S research and operational communities to address the following questions:

What is the current performance level of sub-seasonal precipitation forecasts over land? Over which continental regions can these forecasts be best trusted? How performance levels vary through the seasons of the year?
What is the current capability of S2S models in anticipating the occurrence of extreme precipitation events over land (periods of deficit or excess precipitation)?
How well the main patterns of precipitation variability on the sub-seasonal time scale over various continental regions are represented in S2S prediction models?
How best to combine and calibrate sub-seasonal precipitation forecasts over land in order to produce improved, combined and well-calibrated products and services?
Are there identifiable opportunities for producing sub-seasonal precipitation forecasts over land with improved quality? For example, are forecasts produced during Madden and Julian Oscillation (MJO) and/or El Niño Southern Oscillation (ENSO) events more skilful than when neutral conditions are present? Are forecasts for active and break rainfall phases and dry/wet spells (or other quantities of interest) of adequate quality for developing forecast products for use in application sectors?

In order to address these questions the research and operational communities are encouraged to explore existing and develop novel methodologies for forecast calibration, combination and verification. Following the S2S verification chapter produced by the JWGFVR for the recent S2S book, it is particularly encouraged the identification of the most relevant forecast quality attributes for the target audiences (e.g. model and forecast developers, and various application sectors) in order to choose appropriate scores and metrics to be able to adequately address clearly and previously defined verification questions of interest. This practice helps performing a thorough assessment of sub-seasonal forecasts from both the probabilistic and deterministic points of view.

4) Current work of S2S operational and research communities on calibration, multi-model combination, verification and forecast products generation

4.1) Pilot real-time sub-seasonal multi-model ensemble (MME) forecasts and verification products at the WMO LC-LRFMME

The WMO Lead Centre for Long Range Forecast Multi-Model Ensemble (LC-LRFMME) has been developing a pilot system for real-time multi-model subseasonal forecasts using real-time forecasts (and hindcasts) from a subset of models contributing to the WWRP/WCRP S2S research project accessible via ECMWF data archive. Following this link the S2S research and operational communities have the opportunity to see the characteristics of the pilot real-time sub-seasonal MME prediction system developed by the LC-LRFMME, which includes forecast and verification products. These additional slides provide examples of products developed in support of future global, regional and national operational activities performed by WMO members. Subseasonal models from eight Global Producing Centers (GPCs) are currently used: Beijing, ECMWF, Exeter, Melbourne, Montreal, Seuol, Tokyo and Washington. A range of forecast products has been developed including probabilities for tercile categories of weekly/fortnightly averages of 2m temperature and precipitation as well as the MJO and BSISO indices. Hindcast verification has also been generated using ROC curves and scores, reliability diagrams, root mean square error and correlation between hindcast and observed anomalies.

4.2) IRI initiative on investigating the seasonality of subseasonal rainfall and temperature global prediction skill

This document summarizes the methods followed to conduct a global predictive skill assessment for uncalibrated rainfall and 2-m temperature forecasts, produced using the ECMWF’s IFS model, available through the WWRP/WCRP S2S Prediction Project Database via the International Research Institute for Climate and Society (IRI) Data Library.

4.3) Projects and networks dealing with S2S predictions

ACToday: The Adapting Agriculture to Climate Today, for Tomorrow project.

African SWIFT: Science for Weather Information and Forecasting Techniques.

CLIMAX: Climate Services Through Knowledge Co-Production: A Euro-South American Initiative for Strengthening Societal Adaptation Response to Extreme Events.

CSSP Brazil: Climate Science for Service Partnership Brazil

CSSP China: Climate Science for Service Partnership China

SNAP: Stratospheric Network for the Assessment of Predictability. See verification plans on page 6 of this presentation.

WCSSP India: Weather and Climate Science for Service Partnership India

4.4) Software tools

PyCPT of the International Research Institute for Climate and Society (IRI)

4.5) Web portals

IRI Sub-seasonal forecasts map room

CLIMAX project forecast maps: Precipitation, 2 meter temperature, 200 hPa Geopotential height and Outgoing Longwave Radiation.

5) Publications

Below is a list of publications recently produced by the international S2S research community on forecast verification, calibration and combination, including prediction quality assessment and methodological studies. Additional references including books and technical reports on verification methods relevant for S2S verification are available in section 4 of the phase I S2S verification and products sub-project wiki page. A more comprehensive list of publications produced by the international community including various other S2S research aspects is available in the main S2S project website.

2020

· Dirmeyer, P. A. & Ford, T. W. (2020). A Technique for Seamless Forecast Construction and Validation from Weather to Monthly Time Scales. Mon. Wea. Rev. (2020) 148 (9): 3589–3603. https://doi.org/10.1175/MWR-D-19-0076.1

· Alvarez, M. S., Coelho, C. A. S., Osman, M., Firpo, M. Â. F., & Vera, C. S. (2020). Assessment of ECMWF Subseasonal Temperature Predictions for an Anomalously Cold Week Followed by an Anomalously Warm Week in Central and Southeastern South America during July 2017. Weather and Forecasting, 35(5), 1871-1889.

· Specq, D., Batté, L. (2020). Improving subseasonal precipitation forecasts through a statistical–dynamical approach : application to the southwest tropical Pacific. Clim. Dyn. https://doi.org/10.1007/s00382-020-05355-7

· Phakula, S., Landman, W. A., Engelbrecht, C. J., & Makgoale, T. (2020) Forecast Skill of Minimum and Maximum Temperatures on Subseasonal‐to‐Seasonal Timescales Over South Africa. Earth and Space Science, 7(2), e2019EA000697.

· Manrique-Suñén, A., Gonzalez-Reviriego, N., Torralba, V., Cortesi, N., & Doblas-Reyes, F. J. (2020). Choices in the verification of S2S forecasts and their implications for climate services. Monthly Weather Review, 1-41.

· Wang, Y., Ren, H. L., Zhou, F., Fu, J. X., Chen, Q. L., Wu, J., ... & Zhang, P. Q. (2020). Multi-Model Ensemble Sub-Seasonal Forecasting of Precipitation over the Maritime Continent in Boreal Summer. Atmosphere, 11(5), 515.

· Specq, D., Batté, L., Déqué, M., & Ardilouze, C. (2020). Multimodel forecasting of precipitation at subseasonal timescales over the southwest tropical Pacific. Earth and Space Science, (Accepted)

· Robertson, A. W., Vigaud, N., Yuan, J., & Tippett, M. K. (2020). Towards identifying subseasonal forecasts of opportunity using North American weather regimes. Monthly Weather Review.

· He, H., Yao, S., Huang, A., & Gong, K. (2020). Evaluation and Error Correction of the ECMWF Subseasonal Precipitation Forecast over Eastern China during Summer. Advances in Meteorology, 2020.

· Taguchi, M. (2020). Verification of Subseasonal-to-Seasonal Forecasts for Major Stratospheric Sudden Warmings in Northern Winter from 1998/99 to 2012/13. Advances in Atmospheric Sciences, 37(3), 250-258.

· Phakula, S., Landman, W. A., Engelbrecht, C. J., & Makgoale, T. (2020). Forecast Skill of Minimum and Maximum Temperatures on Subseasonal‐to‐Seasonal Timescales Over South Africa. Earth and Space Science, 7(2), e2019EA000697.

· Son, S. W., Kim, H., Song, K., Kim, S. W., Martineau, P., Hyun, Y. K., & Kim, Y. (2020). Extratropical Prediction Skill of the Subseasonal‐to‐Seasonal (S2S) Prediction Models. Journal of Geophysical Research: Atmospheres, 125(4), e2019JD031273.

· Lin, H. (2020). Subseasonal Forecast Skill over the Northern Polar Region in Boreal Winter. Journal of Climate, 33(5), 1935-1951.