What is Evaluation (AI)?
The systematic measurement of AI system outputs against defined quality criteria, covering accuracy, relevance, safety, and task completion. Evaluations can be automated (test suites with expected outputs or scoring rubrics), human-reviewed, or a hybrid of both. Without ongoing evaluation, there is no reliable way to know whether a model or prompt change made things better or worse.
My perspective
In practice
Evaluation is the discipline most AI teams skip early and regret later. An eval suite is essentially a test suite for model behavior: representative inputs with expected outputs (or rubrics), run automatically so regressions surface immediately. The hardest part is defining what "good" looks like for open-ended tasks; no framework makes that judgment for you. Teams that invest in evals early ship changes faster and with far more confidence.