Highlights: The WWRP/WGNE Joint Working Group on Forecast Verification Research (JWGFVR) is pleased to announce the around-the-clock 2020 International Verification Methods Workshop Online (2020-IVMW-O), which will be held virtually, during two weeks, from the 9th to the 13th and from the 16th to the 20th of November 2020. This workshop will consist in 2-hour online daily sessions, with live-stream presentations and discussion.
The S2S community is cordially invited to submit abstracts on S2S verification methodologies to the workshop. Registration and abstract submission is open at the workshop website https://jwgfvr.univie.ac.at/
Please note the following timeline:
- Abstract submission opens: 30 June 2020
- Registration opens: 30 June 2020
- Abstract submission closed: 7 September 2020
- Registration closes: 30 September 2020
- Notification of Acceptance: 30 September 2020
- Presentation submission: 30 October 2020
- Online Workshop: 9-13 and 16-20 November 2020
Membership of Research to Operations (R2O) and S2S forecast and verification products development sub-project
Caio Coelho (CPTEC/INPE, Brazil)
Andrew Robertson (IRI, USA)
Arun Kumar (NOAA, USA)
Yuhei Takaya (JMA, Japan)
Anca Brookshaw (ECMWF)
Debra Hudson (BoM, Australia)
Angel Muñoz (IRI, USA)
Joanne Robbins (UKMO, UK)
1) Scientific and Operational Objectives
- Pursue research for testing and developing methodologies for calibration, multi-model combination, verification and generation of forecast products.
Coordinate with the relevant WMO technical commissions to define the standards and protocols for operational implementation and exchange of S2S forecasts such that by the end of the Phase II of the S2S, the infrastructure related to the data exchange to support research can be transitioned into the operational domain.
The R2O and S2S forecast and verification products activities plan provides additional information about this sub-project, which builds on the work of the previous (Phase I) S2S Verification and Products sub-project.
2) Linkages with WMO activities
On the research side this sub-project has linkages with the Joint Working Group on Forecast Verification Research (JWGFVR), a WMO joint working group of the Working Group on Numerical Experimentation (WGNE) and the World Weather Research Programme (WWRP). This sub-project also has synergies with the World Climate Research Programme (WCRP) Working Group on Subseasonal to Interdecadal Prediction (WGSIP) through the Climate forecast information for decision making (I4D) project.
On the operational side, this sub-project has linkages with Inter-Programme Expert Team on Operational Predictions from Sub-seasonal to Longer-Time Scale (IPET-OPSLS), a WMO joint team of the Commission for Basic Systems (CBS) and the Commission for Climatology (CCl).
3) Proposed questions to be addressed
The World Weather Research Programme (WWRP) has flagged improving forecasts of precipitation over land as an important area for S2S to focus research and services development efforts. In order to help advance scientific knowledge and the development of forecast and verification products in this priority area this sub-project invites the S2S research and operational communities to address the following questions:
What is the current performance level of sub-seasonal precipitation forecasts over land? Over which continental regions can these forecasts be best trusted? How performance levels vary through the seasons of the year?
What is the current capability of S2S models in anticipating the occurrence of extreme precipitation events over land (periods of deficit or excess precipitation)?
How well the main patterns of precipitation variability on the sub-seasonal time scale over various continental regions are represented in S2S prediction models?
How best to combine and calibrate sub-seasonal precipitation forecasts over land in order to produce improved, combined and well-calibrated products and services?
Are there identifiable opportunities for producing sub-seasonal precipitation forecasts over land with improved quality? For example, are forecasts produced during Madden and Julian Oscillation (MJO) and/or El Niño Southern Oscillation (ENSO) events more skilful than when neutral conditions are present? Are forecasts for active and break rainfall phases and dry/wet spells (or other quantities of interest) of adequate quality for developing forecast products for use in application sectors?
In order to address these questions the research and operational communities are encouraged to explore existing and develop novel methodologies for forecast calibration, combination and verification. Following the S2S verification chapter produced by the JWGFVR for the recent S2S book, it is particularly encouraged the identification of the most relevant forecast quality attributes for the target audiences (e.g. model and forecast developers, and various application sectors) in order to choose appropriate scores and metrics to be able to adequately address clearly and previously defined verification questions of interest. This practice helps performing a thorough assessment of sub-seasonal forecasts from both the probabilistic and deterministic points of view.
4) Current work of S2S operational and research communities on calibration, multi-model combination, verification and forecast products generation
4.1) Pilot real-time sub-seasonal multi-model ensemble (MME) forecasts and verification products at the WMO LC-LRFMME
The WMO Lead Centre for Long Range Forecast Multi-Model Ensemble (LC-LRFMME) has been developing a pilot system for real-time multi-model subseasonal forecasts using real-time forecasts (and hindcasts) from a subset of models contributing to the WWRP/WCRP S2S research project accessible via ECMWF data archive. Following this link the S2S research and operational communities have the opportunity to see the characteristics of the pilot real-time sub-seasonal MME prediction system developed by the LC-LRFMME, which includes forecast and verification products. These additional slides provide examples of products developed in support of future global, regional and national operational activities performed by WMO members. Subseasonal models from eight Global Producing Centers (GPCs) are currently used: Beijing, ECMWF, Exeter, Melbourne, Montreal, Seuol, Tokyo and Washington. A range of forecast products has been developed including probabilities for tercile categories of weekly/fortnightly averages of 2m temperature and precipitation as well as the MJO and BSISO indices. Hindcast verification has also been generated using ROC curves and scores, reliability diagrams, root mean square error and correlation between hindcast and observed anomalies.
4.2) IRI initiative on investigating the seasonality of subseasonal rainfall and temperature global prediction skill
This document summarizes the methods followed to conduct a global predictive skill assessment for uncalibrated rainfall and 2-m temperature forecasts, produced using the ECMWF’s IFS model, available through the WWRP/WCRP S2S Prediction Project Database via the International Research Institute for Climate and Society (IRI) Data Library.
4.3) The Australian Bureau of Meteorology adaptable framework for development and real time production of experimental sub-seasonal to seasonal forecast products
This document describes the new post-processing pipeline developed to add value to sub-seasonal and seasonal forecasts produced by the Australian Bureau of Meteorology.
4.4) Projects and networks dealing with S2S predictions
ACToday: The Adapting Agriculture to Climate Today, for Tomorrow project.
African SWIFT: Science for Weather Information and Forecasting Techniques.
CLIMAX: Climate Services Through Knowledge Co-Production: A Euro-South American Initiative for Strengthening Societal Adaptation Response to Extreme Events.
CSSP China: Climate Science for Service Partnership China
SNAP: Stratospheric Network for the Assessment of Predictability. See verification plans on page 6 of this presentation.
WCSSP India: Weather and Climate Science for Service Partnership India
4.5) Software tools
PyCPT of the International Research Institute for Climate and Society (IRI)
4.6) Web portals
IRI Sub-seasonal forecasts map room
CLIMAX project forecast maps: Precipitation, 2 meter temperature, 200 hPa Geopotential height and Outgoing Longwave Radiation.
5) Publications
Below is a list of publications recently produced by the international S2S research community on forecast verification, calibration and multi-model combination, including prediction quality assessment and methodological studies. Additional references including books and technical reports on methods relevant for S2S verification are available in section 4 of the phase I S2S verification and products sub-project wiki page. A more comprehensive list of publications produced by the international community including various other S2S research aspects is available in the main S2S project website.
2020
· Dirmeyer, P. A., & Ford, T. W. (2020). A technique for seamless forecast construction and validation from weather to monthly time scales. Monthly Weather Review, 148(9), 3589-3603.
· Dirmeyer, P. A. & Ford, T. W. (2020). A Technique for Seamless Forecast Construction and Validation from Weather to Monthly Time Scales. Mon. Wea. Rev. (2020) 148 (9): 3589–3603. https://doi.org/10.1175/MWR-D-19-0076.1
· Alvarez, M. S., Coelho, C. A. S., Osman, M., Firpo, M. Â. F., & Vera, C. S. (2020). Assessment of ECMWF Subseasonal Temperature Predictions for an Anomalously Cold Week Followed by an Anomalously Warm Week in Central and Southeastern South America during July 2017. Weather and Forecasting, 35(5), 1871-1889.
· Specq, D., Batté, L. (2020). Improving subseasonal precipitation forecasts through a statistical–dynamical approach : application to the southwest tropical Pacific. Clim. Dyn. https://doi.org/10.1007/s00382-020-05355-7
· Phakula, S., Landman, W. A., Engelbrecht, C. J., & Makgoale, T. (2020) Forecast Skill of Minimum and Maximum Temperatures on Subseasonal‐to‐Seasonal Timescales Over South Africa. Earth and Space Science, 7(2), e2019EA000697.
· Manrique-Suñén, A., Gonzalez-Reviriego, N., Torralba, V., Cortesi, N., & Doblas-Reyes, F. J. (2020). Choices in the verification of S2S forecasts and their implications for climate services. Monthly Weather Review, 1-41.
· Wang, Y., Ren, H. L., Zhou, F., Fu, J. X., Chen, Q. L., Wu, J., ... & Zhang, P. Q. (2020). Multi-Model Ensemble Sub-Seasonal Forecasting of Precipitation over the Maritime Continent in Boreal Summer. Atmosphere, 11(5), 515.
· Specq, D., Batté, L., Déqué, M., & Ardilouze, C. (2020). Multimodel forecasting of precipitation at subseasonal timescales over the southwest tropical Pacific. Earth and Space Science, (Accepted)
· Robertson, A. W., Vigaud, N., Yuan, J., & Tippett, M. K. (2020). Towards identifying subseasonal forecasts of opportunity using North American weather regimes. Monthly Weather Review.
· He, H., Yao, S., Huang, A., & Gong, K. (2020). Evaluation and Error Correction of the ECMWF Subseasonal Precipitation Forecast over Eastern China during Summer. Advances in Meteorology, 2020.
· Taguchi, M. (2020). Verification of Subseasonal-to-Seasonal Forecasts for Major Stratospheric Sudden Warmings in Northern Winter from 1998/99 to 2012/13. Advances in Atmospheric Sciences, 37(3), 250-258.
· Phakula, S., Landman, W. A., Engelbrecht, C. J., & Makgoale, T. (2020). Forecast Skill of Minimum and Maximum Temperatures on Subseasonal‐to‐Seasonal Timescales Over South Africa. Earth and Space Science, 7(2), e2019EA000697.
· Son, S. W., Kim, H., Song, K., Kim, S. W., Martineau, P., Hyun, Y. K., & Kim, Y. (2020). Extratropical Prediction Skill of the Subseasonal‐to‐Seasonal (S2S) Prediction Models. Journal of Geophysical Research: Atmospheres, 125(4), e2019JD031273.
· Lin, H. (2020). Subseasonal Forecast Skill over the Northern Polar Region in Boreal Winter. Journal of Climate, 33(5), 1935-1951.
· Wang, S. (2020). A precipitation-based index for tropical intraseasonal oscillations. Journal of Climate, 33(3), 805-823.
· Vigaud, N., Tippett, M. K., Yuan, J., Robertson, A. W., & Acharya, N. (2020). Spatial Correction of Multimodel Ensemble Subseasonal Precipitation Forecasts over North America Using Local Laplacian Eigenfunctions. Monthly Weather Review, 148(2), 523-539.
· Materia, S., Muñoz, Á. G., Álvarez-Castro, M. C., Mason, S. J., Vitart, F., & Gualdi, S. (2020). Multimodel Subseasonal Forecasts of Spring Cold Spells: Potential Value for the Hazelnut Agribusiness. Weather and Forecasting, 35(1), 237-254.
2019
· Vigaud, N., Tippett, M. K., & Robertson, A. W. (2019). Deterministic Skill of Subseasonal Precipitation Forecasts for the East Africa‐West Asia Sector from September to May. Journal of Geophysical Research: Atmospheres, 124(22), 11887-11896.
· Vigaud, N., Tippett, M. K., Yuan, J., Robertson, A. W., & Acharya, N. (2019). Probabilistic skill of subseasonal surface temperature forecasts over North America. Weather and Forecasting, 34(6), 1789-1806.
· Lin, H. (2019). Subseasonal Forecast Skill over the Northern Polar Region in Boreal Winter. Journal of Climate, (Accepted)
· Pan, B., K. Hsu, A. AghaKouchak, S. Sorooshian, and W. Higgins (2019). Precipitation Prediction Skill for the West Coast United States: From Short to Extended Range. J. Climate, 32, 161–182, https://doi.org/10.1175/JCLI-D-18-0355.1
· Albers, J. R., & Newman, M. (2019). A Priori Identification of Skillful Extratropical Subseasonal Forecasts. Geophysical Research Letters.
· Li, W., J. Chen, L. Li, H. Chen, B. Liu, C. Xu, and X. Li, (2019): Evaluation and Bias Correction of S2S Precipitation for Hydrological Extremes. J. Hydrometeor., 20, 1887–1906, https://doi.org/10.1175/JHM-D-19-0042.1
· Richardson, D., H. Fowler, C. Kilsby, R. neal and R. Dankers, (2019). Improving sub-seasonal forecast skill of meteorological drought: a weather pattern approach. Nat. Hazards Earth Syst. Sci., https://doi.org/10.5194/nhess-2019-221
· Lee, Seungsoo;Kim, Gayoung;Yoon, Soonjo;An, Hyunuk, (2019). Improvement of precipitation forecasting skill of ECMWF data using multi-layer perceptron technique. Journal of Korea Water Resources Association, Volume 52, Issue 7, pp 475-482. https://doi.org/10.3741/JKWRA.2019.52.7.475
· ZHU, Hanchen and Haishan CHEN and Yang ZHOU and Xuan DONG, (2019). Evaluation of the subseasonal forecast skill of surface soil moisture in the S2S database. Atmospheric and Oceanic Science Letters, 12, 6, 467-474, https://doi.org/10.1080/16742834.2019.1663123
· Kim, H., Janiga, M. A., & Pegion, K. ( 2019). MJO propagation processes and mean biases in the SubX and S2S reforecasts. Journal of Geophysical Research: Atmospheres, 124, 9314– 9331. https://doi.org/10.1029/2019JD031139
· Rao, J., Ren, R., Chen, H., Liu, X., Yu, Y., Hu, J., & Zhou, Y. ( 2019). Predictability of stratospheric sudden warmings in the Beijing Climate Center Forecast System with statistical error corrections. Journal of Geophysical Research: Atmospheres, 124, 8385– 8400. https://doi.org/10.1029/2019JD030900
· Olaniyan, E., Adefisan, E. A., Balogun, A. A., & Lawal, K. A. (2019). The influence of global climate drivers on monsoon onset variability in Nigeria using S2S models. Modeling Earth Systems and Environment, 1-24.
· Kolachian, R., & Saghafian, B. (2019). Deterministic and probabilistic evaluation of raw and post processed sub-seasonal to seasonal precipitation forecasts in different precipitation regimes. Theoretical and Applied Climatology, 137(1-2), 1479-1493.
· Pan, B., Hsu, K., AghaKouchak, A., Sorooshian, S., & Higgins, W. (2019). Precipitation Prediction Skill for the West Coast United States: From Short to Extended Range. Journal of Climate, 32(1), 161-182.
· DeFlorio, M. J., Waliser, D. E., Guan, B., Ralph, F. M., & Vitart, F. (2019). Global evaluation of atmospheric river subseasonal prediction skill. Climate Dynamics, 52(5-6), 3039-3060.
· Miao, Q., Pan, B., Wang, H., Hsu, K., & Sorooshian, S. (2019). Improving Monsoon Precipitation Prediction Using Combined Convolutional and Long Short Term Memory Neural Network. Water, 11(5), 977.
· Pan, B., Hsu, K., AghaKouchak, A., Sorooshian, S., & Higgins, W. (2019). Precipitation Prediction Skill for the West Coast United States: From Short to Extended Range. Journal of Climate, 32(1), 161-182.
· de Andrade, F. M., Coelho, C. A., & Cavalcanti, I. F. (2019). Global precipitation hindcast quality assessment of the Subseasonal to Seasonal (S2S) prediction project models. Climate Dynamics, 52(9-10), 5451-5475.
2018
· Olaniyan E, Adefisan EA, Oni F, Afiesimama E, Balogun AA and Lawal KA (2018). Evaluation of the ECMWF Sub-seasonal to Seasonal Precipitation Forecasts during the Peak of West Africa Monsoon in Nigeria, Front. Environ. Sci. 6:4. doi: 10.3389/fenvs.2018.00004
· Doss-Gollin, J., Muñoz, Á. G., Mason, S. J., & Pastén, M. (2018). Heavy Rainfall in Paraguay during the 2015/16 Austral Summer: Causes and Subseasonal-to-Seasonal Predictive Skill. Journal of Climate, 31(17), 6669-6685.
· Goessling, H. F., & Jung, T. (2018). A probabilistic verification score for contours: Methodology and application to Arctic ice‐edge forecasts. Quarterly Journal of the Royal Meteorological Society, 144(712), 735-743.
· Coelho, C. A., Firpo, M. A., & de Andrade, F. M. (2018). A verification framework for South American sub-seasonal precipitation predictions. Meteorologische Zeitschrift, 503-520.
· DeFlorio, M. J., Waliser, D. E., Guan, B., Lavers, D. A., Ralph, F. M., & Vitart, F. (2018). Global assessment of atmospheric river prediction skill. Journal of Hydrometeorology, 19(2), 409-426.
· Ferranti, L., Magnusson, L., Vitart, F., & Richardson, D. S. (2018). How far in advance can we predict changes in large‐scale flow leading to severe cold conditions over Europe?. Quarterly Journal of the Royal Meteorological Society, 144(715), 1788-1802.
· Ford, T. W., Dirmeyer, P. A., & Benson, D. O. (2018). Evaluation of heat wave forecasts seamlessly across subseasonal timescales. npj Clim. Atmos. Sci., 1, 20.
· Gao, M., Wang, B., Yang, J., & Dong, W. (2018). Are peak summer sultry heat wave days over the Yangtze–Huaihe River basin predictable?. Journal of Climate, 31(6), 2185-2196.
· Janiga, M. A., J. Schreck III, C., Ridout, J. A., Flatau, M., Barton, N. P., Metzger, E. J., & Reynolds, C. A. (2018). Subseasonal forecasts of convectively coupled equatorial waves and the MJO: Activity and predictive skill. Monthly Weather Review, 146(8), 2337-2360.
· Karpechko, A. Y., Charlton‐Perez, A., Balmaseda, M., Tyrrell, N., & Vitart, F. (2018). Predicting Sudden Stratospheric Warming 2018 and Its Climate Impacts With a Multimodel Ensemble. Geophysical Research Letters, 45(24), 13-538.
· Kolachian, R., & Saghafian, B. (2018). Deterministic and probabilistic evaluation of raw and post processed sub-seasonal to seasonal precipitation forecasts in different precipitation regimes. Theoretical and Applied Climatology, 1-15.
· Lim, Y., Son, S. W., & Kim, D. (2018). MJO prediction skill of the subseasonal-to-seasonal prediction models. Journal of Climate, 31(10), 4075-4094.
· Nardi, K. M., Barnes, E. A., & Ralph, F. M. (2018). Assessment of numerical weather prediction model reforecasts of the occurrence, intensity, and location of atmospheric rivers along the West Coast of North America. Monthly Weather Review, 146(10), 3343-3362.
· Vigaud, N., Tippett, M. K., & Robertson, A. W. (2018). Probabilistic Skill of Subseasonal Precipitation Forecasts for the East Africa–West Asia Sector during September–May. Weather and Forecasting, 33(6), 1513-1532.
· Vuillaume, J. F., Dorji, S., Komolafe, A., & Herath, S. (2018). Sub-seasonal extreme rainfall prediction in the Kelani River basin of Sri Lanka by using self-organizing map classification. Natural Hazards, 94(1), 385-404.
· Zhou, Y., Yang, B., Chen, H., Zhang, Y., Huang, A., & La, M. (2018). Effects of the Madden–Julian Oscillation on 2-m air temperature prediction over China during boreal winter in the S2S database. Climate Dynamics, 1-19.
2017
· Vitart, F. (2017). Madden—Julian Oscillation prediction and teleconnections in the S2S database. Quarterly Journal of the Royal Meteorological Society, 143(706), 2210-2220.
· Bombardi RJ, Pegion KV, Kinter JL, Cash BA and Adams JM (2017): Sub-seasonal Predictability of the Onset and Demise of the Rainy Season over Monsoonal Regions. Front. Earth Sci. 5:14. doi: 10.3389/feart.2017.00014
· Ferrone, A., D. Mastrangelo, and P. Malguzzi. (2017) Multimodel probabilistic prediction of 2 meter-temperature anomalies on the monthly timescale. Advances in Science and Research 14, 123-129. Online publication date: 8-May-2017.
· Schiraldi, N.J. and P.E. Roundy, 2017: Seasonal-to-Subseasonal Model Forecast Performance during Agricultural Drought Transition Periods in the U.S. Corn Belt, 2017. Mon. Wea. Rev, 145, 3687-3708. DOI: 10.1175/MWR-D-17-0026.1
· Vigaud, N., A. W. Robertson and M. K. Tippett, 2017a: Multi-model ensembling of subseasonal precipitation forecasts over North America, Mon. Wea. Rev., 145, 3913-3928.
· Ichikawa, Y. and M. Inatsu, 2017: An alternative estimate of potential predictability on the Madden–Julian oscillation phase space using S2S models. Atmosphere 2017, 8(8), 150; doi:10.3390/atmos8080150
· Liu, X., Wu, T., Yang, S., Li, T., Jie, W., Zhang, L., ... & Ren, H. (2017). MJO prediction using the sub-seasonal to seasonal forecast model of Beijing Climate Center. Climate Dynamics, 48(9-10), 3283-3307.
· Vigaud, N., Robertson, A. W., Tippett, M. K., & Acharya, N. (2017). Subseasonal predictability of boreal summer monsoon rainfall from ensemble forecasts. Frontiers in Environmental Science, 5, 67.
2016
· Lowe, R., M. García-Díez, J. Ballester, J. Creswick, J.-M. Robine, F. R. Herrmann, and X. Rodó, 2016: Evaluation of an Early-Warning System for Heat Wave-Related Mortality in Europe: Implications for Sub-seasonal to Seasonal Forecasting and Climate Services, Int J Environ Res Public Health. 13(2): 206
