AI Research & Knowledge
Make the comparison understandable before interpreting it
This record method follows a fictional team comparing document-summary assistants. It separates the task, the conditions and the review, so a useful result stays attached to the circumstances that produced it. The worksheet is a planning aid, not a universal intelligence test.
01
Define the task and the information supplied
Write the question the assistant must answer and identify the exact source set. In the fictional example, the task is to summarise one project decision using a dated meeting note and its later correction. Save the task wording and document versions together. State whether retrieval, external tools or human hints are allowed. A change in those conditions creates a different comparison, even if the final answers look similar. Keep a blank field for unresolved setup questions rather than quietly filling them with assumptions after the exercise has begun.
02
Describe the review rule and retain failures
Decide what a satisfactory summary must include before examining the answers. For this example, the reviewer checks whether the later correction is reflected and whether each substantive point can be traced to the supplied material. Save the answer and the reviewer’s explanation, including omissions or unsupported additions. If an answer is corrected by a person, record that intervention separately. A successful example and a failure both belong in the record. That makes it possible to discuss the observed work without presenting a selected demonstration as a general conclusion.
03
Keep performance, breadth and authority separate
Close with three distinct notes: what happened on this task, which other situations remain unexamined and what actions the system was permitted to take. Producing a useful summary does not by itself justify sending it to colleagues automatically. For the fictional team, a person could review the draft before any communication. The record should also identify the model or product version and the date of the comparison. When conditions change, a later review can revisit the narrow conclusion without treating an earlier result as permanent evidence of broad capability.