Recent advances in Generative AI capabilities for natural language understanding and text generation are driving the creation of automated question-answering services for production agriculture. Ensuring the reliability and accuracy of the advice from these systems is paramount because incorrect responses to crop production questions can lead to serious consequences for farm incomes, soil health, and the sustainability of family farming operations. As such, the Center for Digital Agriculture (CDA) at the University of Illinois Urbana-Champaign has announced a new consortium, AI AgriBench, to evaluate and build confidence in AI-driven question-answering systems supporting farmers, agronomists, industry, and the broader agriculture community.
The overarching goal of this effort is to provide farmers, policymakers, and the public with trustworthy mechanisms to evaluate and achieve high confidence in the new generation of AI-driven agronomy tools. We need to ensure these services deliver on their promise of valuable and accurate technical information to farmers and ag professionals worldwide.
The new public consortium led by the CropWizard project within the Center for Digital Agriculture will create and oversee the benchmarking effort. In addition to the U. of I., the consortium’s founding members include Bayer Crop Science, KissanAI, and the Extension Foundation. The consortium will be open to any organizations interested in contributing their expertise to support the creation, monitoring, and long-term maintenance of this benchmarking effort.
This consortium brings together deep technical expertise in both AI and agronomy across leading academic and industry groups to tackle the challenges these new systems bring. Recent efforts have led to open source systems like CropWizard and proprietary commercial systems, all of which are designed for answering technical questions about agronomy for farmers, agronomists, scientists, and other agriculture professionals. Extensive data sets and software tools within these organizations provide the foundation for building reliable, automated evaluation pipelines based on expert-validated ground truth answers.
Over the next few months, the consortium will convene members and maintain a leaderboard on a publicly accessible, secure website. This website will be available to any groups wishing to evaluate their models and publish their scores. For full transparency, a detailed description of the evaluation methodology, data sets, and governance process will also be publicly available. An independent oversight board administered by CDA will supervise and advise the consortium’s operations.
If you are interested in joining a mailing list to be informed about the consortium’s plans or becoming a contributing consortium member, please go to https://go.illinois.edu/aiagribench-form to sign up. We welcome interested parties with relevant data sets, expertise, or other valuable resources to participate and contribute.