Crop Model Calibration

Main Contacts for Initiative

Daniel Wallach (Daniel.Wallach@inrae.fr)

Sabine Seidel (sabine.seidel@uni-bonn.de)

Peter Thorburn (Peter.Thorburn@csiro.au)

Brief description of activity

The calibration activity has three interrelated objectives. 1) Build a knowledge base concerning calibration practices for crop models. 2) Develop and test guidelines for improved calibration practices. 3). Develop tools for application of improved calibration practices.

Phase 1 was a web-based survey of calibration practices conducted 2016-2017. In phase 2 (2018-2020), 30 participating modelling groups applied their usual calibration practices to two data sets of phenology data, one for wheat in France and the other for wheat in Australia. In each case, participants were provided with phenology data from one set of environments (the calibration environments) and asked to provide simulated values for a different set of environments (the evaluation environments). The purpose here was to document usual practices and to provide a baseline of simulation accuracy, for comparison with new guidelines.

Phase 3 (2020-2021), involves developing guidelines for improved calibration practices and then going beyond recommendations to propose a detailed protocol for calibration using phenology data. The protocol has been applied by multiple modelling groups using the same data as in phase 2, and prediction accuracy will be compared between usual methods and the proposed protocol. Two calibration tools were developed in this phase; a set of tables to systematize and document the application of the protocol to each specific model, and a software tool for doing the calibrations of the protocol.

In phase 4 (starting mid 2021), the approach of phase 3 is extended to the general case where multiple types of data are available for calibration, including phenology data, yield and yield components and in-season biomass measurements. A protocol for this general case will be developed and tested.

In this project of the uncertainty theme, we aim to identify promising approaches to crop model calibration, apply them to multiple models using a common data set and evaluate them. We will also consider how to evaluate the calibrated model, and how to quantify the relation of prediction error to amount of data.

Overview of Participants

Twenty-nine international modeling groups, using 23 different model structures, participated in phase 2.

The first phase was a survey with over 200 participants. The call for the second phase, open to all, is now being prepared.

Boxplots of skill scores for prediction of days to Zadoks stages Z30, Z65, and Z90, and averaged over stages (all) for the evaluation data. Skill score is 1 for a modeling group that predicts perfectly, and is less than or equal to 0 for a modeling group that does no better than using average days to each stage in the calibration data (EF skill score) or than using the average number of degree days to each stage in the calibration data (skillT skill score). For readability the y axis is cut off at –1.

Current Research Focus

Phase 2 aims to test a detailed protocol for calibration using phenology data. Two major challenges are addressed here. First, can one develop a protocol which provides detailed instructions as to how to carry out calibration, but is nonetheless applicable to a wide range of models? The second challenge is to apply the protocol in practice with multiple teams using different models.

Major research questions are how the protocol implementation differs from the usual methods, whether the proposed protocol improves prediction accuracy compared to each group’s usual approach, and the effect of using a standardized calibration approach on multi-model ensemble results.

Phase 1 aimed to test calibration methods on phenology models.

Recent Noteworthy Finding

We evaluated how well 28 crop modeling groups simulate wheat phenology in Australia, in the case where the calibration data and the evaluation data were representative of the same population but had neither sites nor years in common, so that this was a rigorous test of how well crop models simulate phenology for new environments. Mean absolute error for the evaluation data ranged from six to 20 days depending on the modeling group, with a median of 9 days. About two thirds of the modeling groups performed better than a simple but relevant benchmark, which predicts phenology assuming a constant temperature sum for each development stage. The added complexity of crop models beyond just the effect of temperature is therefore justified in most cases. As found in many other studies, the multi-modeling group mean and median had prediction errors nearly as small as the best modeling group, suggesting that using these ensemble predictors is a good strategy. Prediction errors for calibration and evaluation environments were found to be significantly correlated, which suggests that for interpolation type studies, it would be of interest to test ensemble predictors that weight individual models based on performance for the calibration data.

The main result of the Phase 1 survey was to highlight the large diversity of approaches to crop model calibration by practitioners.

Recent Papers/Reports

(Seidel et al., 2018; Wallach et al., 2021c, 2021b, 2021a)

Seidel, S.J., T. Palosuo, P. Thorburn, and D. Wallach, 2018: Towards improved calibration of crop models – Where are we now and where should we go? European Journal of Agronomy, Volume 94, March 2018, Pages 25-35, https://doi.org/10.1016/j.eja.2018.01.006.

Wallach, D., Palosuo, T., Thorburn, P., Gourdain, E., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hochman, Z., Hoek, S., Hoogenboom, G., Horan, H., Huang, M., Jabloun, M., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Nariman Zadeh, H., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., 2021a. How well do crop modeling groups predict wheat phenology, given calibration data from the target population? Eur. J. Agron. 124, 126195. https://doi.org/https://doi.org/10.1016/j.eja.2020.126195

Wallach, D., Palosuo, T., Thorburn, P., Hochman, Z., Andrianasolo, F., Asseng, S., Basso, B., Buis, S., Crout, N., Dumont, B., Ferrise, R., Gaiser, T., Gayler, S., Hiremath, S., Hoek, S., Horan, H., Hoogenboom, G., Huang, M., Jabloun, M., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.C., Launay, M., Lewan, E., Luo, Q., Maestrini, B., Moriondo, M., Padovan, G., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., Wit, A. de, Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., 2021b. Multi model evaluation of phenology prediction for wheat in Australia. Agric. For. Meteorol. in press. https://doi.org/10.1101/2020.06.06.133504

Wallach, D., Palosuo, T., Thorburn, P., Hochman, Z., Gourdain, E., Andrianasolo, F., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hiremath, S., Hoek, S., Horan, H., Hoogenboom, G., Huang, M., Jabloun, M., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Lewan, E., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Zadeh, H.N., Padovan, G., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., Wit, A. de, Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., 2021c. The chaos in calibrating crop models. bioRxiv 2020.09.12.294744. https://doi.org/10.1101/2020.09.12.294744