Protect a business rule on pull requests In Factory settings → Proof runs → Incident checks, select the repository and connect CI. Store CODEBOOK_INCIDENT_TOKEN in a protected reporting job. Set CODEBOOK_ORIGIN to your CodeBook web origin. Record the original incident, the full broken and fixed Git commits, and the actual and expected values. The person who records the rule owns its approval and retirement. Example: refund retries should issue exactly one refund. Use a test deployment with synthetic data. Pin a reviewed copy of /runner/incident-check.mjs and a probe file in your CI configuration. Keep the verifier and probe independent of the proposed change, along with the reporting job and its credential. A protected CI job must run the measurements itself and carry the artifact to the reporting step. Do not accept observations or URLs supplied by the pull request author. After saving the incident, open “Build the CI probe” on its rule. Add the requests that reproduce the incident, choose the value to measure, then click Download probe. For a repository with several rules, copy each downloaded entry into the same probe file's “cases” object. Keep each case unchanged and use the same revision path. The required PR check measures and reports all active rules together. An example probes.json for an API that exposes refunds and their count: { "revision_path": "/version", "cases": { "refund.count": { "steps": [ {"method":"POST","path":"/refunds","body":{"request_id":"{{run}}"},"status":201}, {"method":"POST","path":"/refunds","body":{"request_id":"{{run}}"},"status":200}, {"path":"/refunds/by-request/{{run}}"} ], "pointer": "/count" } } } Adapt the paths, payload and expected HTTP statuses to your test service. By default the last step reads JSON. Its pointer selects the value that CI compares as text. Each case gets a fresh random {{run}} value. To catch an endpoint that used to return HTTP 500 and should return 204, use {"steps":[{"path":"/export"}],"observe":"status"}. Enter 500 as the broken value and 204 as the expected value. This mode captures the final HTTP status, including redirects, without following them. Leave status off the final step. To compare a short plain-text result, use {"steps":[{"path":"/invoice/state"}],"observe":"text"}. The check compares the response text after stripping one final line ending. Earlier steps and text or JSON responses use the specified status (200 by default). The revision path must return the full Git SHA of the running deployment as plain text. The runner checks it before and after measurement. If historical revisions lack a version route, your trusted deployment can serve the revision of the running artifact through the same test-service origin. Use build or deployment metadata for that artifact. Keep the same probe on all revisions. Changing a case's steps or observation requires fresh baselines and approval. For each historical revision, provision the test service at that exact commit. Use Node 22 or newer. Supply an unused output filename for every measurement. Here SERVICE_URL, REPO and SHA come from trusted CI deployment metadata. node incident-check.mjs measure probes.json "$SERVICE_URL" "$REPO" "$SHA" result.json node incident-check.mjs report result.json "$SHA" 12 Replace 12 with the ID CodeBook shows for your rule. Both commands must exit 0: on the broken commit the original mistake must recur, and on the fixed commit the expected value must appear. Refresh evidence in CodeBook and approve the check. The stored case includes the digest of the probe used on both revisions. Run the same check on every pull request, using its head SHA. In a separate protected report step consume only the artifact from that run: node incident-check.mjs measure probes.json "$SERVICE_URL" "$REPO" "$SHA" result.json node incident-check.mjs report result.json "$SHA" Make this job required in the repository's merge settings for human and agent pull requests. Its log shows the measured commit, expected value, observation and verdict. A missing value, changed probe, conflicting measurements or a CodeBook outage fail the job. CodeBook also checks the stored result before it automatically merges its own work. If the connection changes, update the protected job with a replacement CI credential from the same settings screen. To protect a field that disappeared in the broken version, choose JSON field exists in the probe builder and supply its JSON pointer. The probe measures false when the field is absent and true when present, including a null value.