AgMIP Data Interoperability Group (AgDIG)

Prior Updates 

Leaders and main contacts for initiative: 

Cheryl Porter, University of Florida, USA
Medha Devare, International Food Policy Research Institute, USA

The AgMIP Data Interoperability Group (AgDIG) was convened during the June 2016 AgMIP6 Global Workshop in in Montpellier, France to address key challenges to data accessibility and interoperability that form the basis for model-based analyses. AgDIG brings together an international team of agricultural production modelers with data and IT experts to guide the course of AgMIP efforts in making data findable, accessible, interoperable and re-useable for modeling activities.

Background and Motivation

Model development and assessment are often impeded by the lack of availability of high quality data in formats that can be combined across locations and years. Key challenges include inconsistent annotation of datasets, making it difficult to assemble, clean, and re-format datasets for re-use in quantitative analyses. Further, measured values are typically not linked with protocols, and variables proliferate—often to unreasonable degrees (e.g., scale differences for disease scoring by different researchers). Many of these concerns will increase in importance as we move from direct observation to obtaining measurements via proximal sensing, with costs associated with harmonizing data after collection becoming more prohibitive. A concerted community approach within and beyond the AgMIP community – including key players such as CGIAR, USDA, INRA, CIRAD, GODAN, FAO, professional societies and others – is needed to foster the consistent use of accepted standards and approaches for annotating and archiving data, including: metadata, ontologies, and data dictionaries. The AgDIG provides a tent under which these efforts can prosper in collaboration, with shared learning and effort.

AgMIP Data Interoperability

Development of a data interoperability system for use by AgMIP Regional Research Teams began in 2011 for the purpose of automating data provisioning for ensemble modeling activities. The AgMIP data interoperability system consists of the ACE data schema (Porter et al., 2015) which uses a data dictionary based on published ICASA standards (White et al., 2014), and a library of translators to harmonize data from diverse sources and to produce crop model-ready datasets for multiple models. The system was developed as an international collaborative effort of more than 30 crop modelers and software developers representing 13 crop simulation models and has been implemented in desktop applications, high performance cloud-based computing frameworks and database APIs.

The AgDIG will promote use of the AgMIP data interoperability system while expanding its current capacity to exchange data with other international data systems to increase the availability of high quality, accessible and interoperable agricultural research data for use in modeling and other quantitative analyses.

Objectives

1. Expand use and development of AgMIP data interoperability tools which allow exchange and re-use of data for agricultural modeling, including formalizing data mapping between ICASA variables and ontology terms recognized by other groups (e.g., INRA, CGIAR, GODAN, Plant Ontology).
2. Expand the current AgMIP Data Interoperability System to include additional domains such as Livestock and Grazing, Pest and Diseases, Farming System Models, Socio-economic modeling, etc.
3. Perform a landscape analysis to understand ongoing related efforts that can be leveraged (e.g. ANAEE, GODAN). Bring together different current and potential key collaborators and efforts in a more formal way to get beyond immediate and traditional research areas.
4. Develop formal links with large infrastructural efforts (e.g. CGIAR Big Data Platform, CyVerse)
5. Develop use cases including LTAR, AgMIP, variety trials, high-throughput phenotyping, gene-based modeling, statistical modeling, LCA, and meta-analysis to identify roadblocks and solutions to enable data-model integration and analytics.
6. Suggest and contribute to proposals funding the development of data interoperability systems and open data access for agricultural research, emphasizing distributed funding sources.

Key deliverables

(first year – July 2016 – June 2017)

1. Completed mapping of ICASA with AgrO and CropOntology
2. Paper(s) drafted on AgMIP data alignment methods and findings
3. User feedback on early beta version of Agronomy Management System and field book
4. AgMIP data translator developed for detailed, multi-factor field crop experimental data using USDA China Wheat data as a use case.
5. Meetings/workshops:
   ‒ Data Management session at ICRISAT
   ‒ Sponsored workshop

Membership

Membership is open to anyone who is interested in contributing to or guiding the future of AgMIP Data Interoperability systems. All members are expected to contribute by providing data, vocabularies or ontologies, or data translation tools. Please contact it@agmip.org if you are interested in participating.

Additional background material

1. Porter, C.H., C. Villalobos, D. Holzworth, R. Nelson, J.W. White, I.N. Athanasiadis, S. Janssen, D. Ripoche, J. Cufi, D. Raes, M. Zhang, R. Knapen, R. Sahajpal, K.J. Boote, J.W. Jones. 2014. Harmonization and translation of crop modeling data to ensure interoperability. Environmental Modelling and Software. 62:495-508.
2. White, J.W., Hunt, L.A., Boote, K.J., Jones, J.W., Koo, J., Kim, S., Porter, C.H., Wilkens, P.W., Hoogenboom, G. 2013. Integrated Description of Agricultural Field Experiments and Production: the ICASA Version 2.0 Data Standards. Computers and Electronics in Agriculture. 96:1-12.
3. Information for researchers on the development efforts of the AgMIP IT team.
4. Information for developers on the software development efforts of the AgMIP IT team.
5. GitHub repository of AgMIP software developed for data interoperability