#evals

A critical breakdown of why statistical AI text classifiers suffer from high false-positive rates on formal, non-native, and technical English.

• Explains why perplexity and burstiness metrics fail in real-world adversarial settings.

Detailed taxonomy of autonomous scientific research agents and the meta-harnesses required to validate hypotheses, run experiments, and synthesize literature.

• Bridges theoretical agent architectures with practical laboratory workflows.

Lilian Weng’s comprehensive guide on building robust evaluation harnesses that allow AI agents to iteratively improve their own performance.

• Focuses on environment feedback loops, automated failure analysis, and benchmark integrity.

Lee Han Chung examining the operational friction of testing LLM agents in CI/CD.

• Addresses sandbox state contamination, flaky non-deterministic tool mockings, and cost bloat.

Eugene Yan on designing automated evaluation environments for offensive and defensive AI security agents.

• Emphasizes capturing exploit mechanics, deterministic scoring sandboxes, and defense-in-depth metrics.

Cameron Wolfe’s extensive breakdown on constructing agent benchmarks.

• Evaluates unit test assertions, trajectory fidelity, tool argument correctness, and human-in-the-loop rubrics.