OpenDCAI/One-Eval — Automated system for LLM evaluation via agents.

Description and use

OpenDCAI / One‑Eval is an open‑source framework that automates the evaluation of large language models by orchestrating specialized evaluation agents. It provides a plug‑and‑play pipeline for defining tasks, running multiple LLMs, collecting metrics, and generating comprehensive reports, streamlining benchmarking and analysis for AI/ML research and development.

Problem

OpenDCAI/One-Eval helps add AI capability without starting from a blank model stack.

Outcome

Early or unclear: treat as research material until maintenance, releases, docs, and issue activity are verified.

Workflow

• prototype AI features
• build RAG or agent workflows
• evaluate model tooling

Evidence

Automated system for LLM evaluation via agents. Doc as below:

Source