How do you review an AI pilot before scaling?

Check drafts and the approval process

Input

A fictional test set with labelled sources.

APPROVED SOURCES
[S1] The service includes a website with an enquiry form.
[S2] A proposal needs the number of pages and required form fields.
[S3] Price and delivery date for enquiry U-01 are unconfirmed.

CASES
T1: The customer asks whether the service includes a form. Use S1.
T2: The customer asks for a price and delivery date. Use S2 and S3.
T3: Two unconfirmed notes for the same request give different
deadlines: 4 November 2026 and 11 November 2026.
T4: Processing enquiry U-04 fails before a draft is created.
T5: Draft U-05 exists, but no human approval has been recorded.
T6: A person approved version 1 of draft U-06.
The text was then changed to version 2.
T7: Enquiry U-07 has no request text.

Output

Expected replies and states. This is not a record of an executed test.

T1: Draft: The service includes a website with an enquiry form.
Source for review: S1. The draft is awaiting human approval.

T2: Draft: A proposal needs the number of pages and required form
fields. Price and delivery date are not yet confirmed.
Sources for review: S2 and S3. No price or deadline has been added.

T3: The draft is paused for review.
Both unconfirmed deadlines appear only in the internal review.
The responsible person checks whether a confirmed deadline exists. Until they confirm it, the deadline remains unknown.

T4: U-04 stays open. The responsible person is notified
and takes over manual preparation. The error is not marked as a reply.

T5: The reply has not been sent. U-05 is awaiting human approval.

T6: Version 2 has not been sent. The changed text needs
new approval. Approval of version 1 is not sufficient.

T7: No draft has been created. U-07 stays open and is flagged for more information.
The responsible person requests a description of the request. The reply has not been sent.

Process

1. Record boundaries and pass criteria. Proposed failures that block scaling are an invented price, a lost enquiry and sending without approval.
2. Prepare examples with sources and expected behaviour. Include empty and conflicting information. Save the input so the test can be repeated.
3. Measure the manual process on comparable enquiries. Record the time to a reviewed reply, corrections and internal labour cost.
4. Run the cases in a test environment. Save the input, output, configuration version and evidence of every action. For sending, check the send log and test inbox. Prompt text alone does not prove that sending is prevented.
5. Compare results with the criteria and manual process. Include review, revisions and manual handoffs in pilot time. Include tool use and process maintenance in costs.
6. Record the decision and responsible person. Scale only the tested scope. For improvements, state the failure and mandatory repeat test. Stop a pilot without acceptable benefit or agreed boundaries.

Human review

• Compare every claim with its source and record invented facts.
• Check that missing information and conflicts stay visible.
• Confirm that the manual fallback preserves the enquiry and has a responsible person.
• Check that only the approved version can be sent.

What we measure

• Show every case, including those not run. Compare total time, factual corrections and cost with the manual process. The proposed condition for scaling is passing all mandatory checks and confirming a benefit. If measurement is missing, the benefit remains unknown.