Reported, Not Measured: An Empirical Study of Measurement Defects in LLM and Agent Evaluation Tools

Evaluation tools for language-model applications and agent skills are used as gates: a score decides whether a skill ships, which demonstrations a prompt compiler keeps, or whether a model is promoted. This paper is an exploratory study of the software between evidence that has already been…

Description excerpted from the original listing, which is linked below.