TypeSafe AI· X：TypeSafe AI (@typesafeai)·· 4 小时前AI 评分22
谁来评测评测者？好评测的标准
AI 导读
关于"谁来评测评测者"，文章提出好评测应具备三个特征：短暂性，即只对某一时刻的能力做快照；有依据，直接测量你关心的任务和数据上的表现；具体，针对期望结果本身而非其代理指标。
正文
who benchmarks the benchmarkers?
a good eval is:
- ephemeral. it snapshots a moment in time and capabilities.
- grounded. it directly measures performance on tasks and data you care about.
- specific. it operates on the desired results and not some proxy of them.
take notes!
来源：TypeSafe AI · x.com