Continuous Feedback and Improvement › Glossary

AI Critic: A second AI model used to evaluate the quality of another AI's outputs. Acts as an automated judge to help scale evaluation efforts.

Error Frequency: The rate at which specific types of errors occur over time. Tracking this metric helps identify which problems are most common and whether improvements are reducing errors.

Feedback Loop: A cyclical process where you use evaluation results to identify weaknesses, make improvements, test them, deploy, and then gather new evaluation data. The foundation of continuous improvement.

Pattern Analysis: The process of reviewing multiple failures or issues to identify common categories or themes, rather than treating each problem as unique. Helps prioritize improvement efforts.

Regression Testing: Running existing test cases after making changes to ensure that fixes for new problems didn't break functionality that was previously working.

Success Rate: The percentage of agent interactions that complete successfully, typically measured through user feedback or task completion metrics. A key indicator of overall agent performance.