评测 Run 血缘模板

适用场景

用于记录一次评测执行涉及的所有版本依赖，保证结果可复现、可对比、可审计。

字段说明

本模板对应第 18.7 和 18.10 节。Run 只保存血缘和聚合摘要；逐 Case 事实与逐 Evaluator 判断分别落入 Case Result、Evaluation Result。下列记录结构不能合并成一个全局 status。

模板

run_id:
task_id:
trigger:
scope:
versions:
  model:
  system_prompt:
  app_prompt:
  dataset:
  evaluator:
  rubric:
  knowledge_base:
  index:
  embedding_model:
  tool_schema:
  agent_config:
  environment:
execution_config:
  runner_version:
  random_seed:
  sampling_config:
    temperature:
    top_p:
    max_tokens:
  concurrency:
  timeout:
  retry_policy:
result_summary:
  total_cases:
  pass_rate:
  high_risk_failures:
  estimated_cost:
    amount:
    currency:
    pricing_version:
  latency:
timestamp:
case_snapshot:
result_hash:
result_storage:
  case_results_uri:
  evaluation_results_uri:
case_result_record:
  case_result_id:
  run_id:
  case_id:
  case_version:
  execution_status: pending | running | completed | skipped | error
  quality_status: passed | failed | needs_human_review | not_evaluated
  input_snapshot_uri:
  output:
  trace_id:
  artifact_refs: []
  execution_error:
    type: none | system_error | timeout | environment_error
    message:
  started_at:
  completed_at:
evaluation_result_record:
  evaluation_result_id:
  case_result_id:
  evaluator_id:
  evaluator_version:
  metric_values: {}
  outcome: passed | failed | needs_human_review | error
  reason:
  evidence_refs: []
  confidence:
    value:
    semantics:
  review_required:
  error_type: none | evaluator_error | invalid_evidence
  evaluated_at:
artifacts:
  report:
  traces:
  logs:
owner:

填写示例

以下数值和版本号仅用于演示字段填写，不代表生产规模、质量结果或通用阈值。

run_id: run_refund_release_20260708_001
task_id: task_refund_release_v12
trigger: refund_prompt_v12 发布前评测
scope: 客服 Agent 退款场景
versions:
  model: model_a_202607
  system_prompt: system_prompt_v5
  app_prompt: refund_prompt_v12
  dataset: refund_eval_set_v8
  evaluator: refund_judge_v3
  rubric: refund_rubric_v4
  knowledge_base: refund_kb_20260701
  index: refund_index_20260701_02
  embedding_model: embedding_v1
  tool_schema: refund_tool_schema_v4
  agent_config: refund_agent_config_v6
  environment: eval_env_202607
execution_config:
  runner_version: eval_runner_v6
  random_seed: 42
  sampling_config:
    temperature: 0.2
    top_p: 0.9
    max_tokens: 2048
  concurrency: 20
  timeout: 60s
  retry_policy: retry_once_on_timeout
result_summary:
  total_cases: 1200
  pass_rate: 0.982
  high_risk_failures: 0
  estimated_cost:
    amount: 86.4
    currency: CNY
    pricing_version: refund_agent_pricing_202607
  latency: p95_7s
timestamp: 2026-07-08T09:30:00Z
case_snapshot: artifact://run_refund_release_20260708_001/cases.jsonl
result_hash: sha256:example_result_hash
result_storage:
  case_results_uri: artifact://run_refund_release_20260708_001/case-results.jsonl
  evaluation_results_uri: artifact://run_refund_release_20260708_001/evaluation-results.jsonl
case_result_example:
  case_result_id: case_result_refund_20260708_017
  run_id: run_refund_release_20260708_001
  case_id: refund_response_clarity_017
  case_version: v3
  execution_status: completed
  quality_status: failed
  input_snapshot_uri: artifact://run_refund_release_20260708_001/inputs/017.json
  output: "请按页面提示操作。"
  trace_id: trace_refund_20260708_017
  artifact_refs:
    - artifact://run_refund_release_20260708_001/outputs/017.json
  execution_error:
    type: none
    message:
  started_at: 2026-07-08T09:31:12Z
  completed_at: 2026-07-08T09:31:15Z
evaluation_result_example:
  evaluation_result_id: eval_result_refund_20260708_017_clarity
  case_result_id: case_result_refund_20260708_017
  evaluator_id: refund_rubric_judge
  evaluator_version: v3
  metric_values:
    answer_clarity: 1
  outcome: failed
  reason: 回答没有说明所需页面、操作步骤或失败后的处理路径
  evidence_refs:
    - artifact://run_refund_release_20260708_001/outputs/017.json
  confidence:
    value: high
    semantics: Judge 自报信号，仅用于复核分流
  review_required: false
  error_type: none
  evaluated_at: 2026-07-08T09:32:01Z
artifacts:
  report: artifact://run_refund_release_20260708_001/report
  traces: artifact://run_refund_release_20260708_001/traces
  logs: artifact://run_refund_release_20260708_001/logs
owner: evalops_owner

常见误区