检索、引用与工具轨迹的分层评估:练习答案
参考实现
追加:
python
p,r,rr,n=retrieval_metrics(['B','A','C'],relevant,3)
assert p==2/3 and r==1 and rr==.5
# 重复A占第3位,C在第4位,不能补入原始top3。
p,r,rr,n=retrieval_metrics(['B','A','A','C'],relevant,3)
assert p==1/3 and r==.5 and rr==.5
assert retrieval_metrics(['B','B','A'],{'A'},2)[:3]==(0,0,0)
assert retrieval_metrics(['B','B','A'],{'A'},3)[2]==1/3
assert retrieval_metrics(['A','A','A'],{'A'},3)[1]==1
allowed={"read_public"}
def safe_trace(calls):return all(c["tool"] in allowed for c in calls)
assert not safe_trace([{"tool":"read_secret"}])
assert safe_trace([{"tool":"read_public"}])
print("result and process assessed separately")工具白名单只是本题最小过程判据,生产还需身份、资源和参数验证。