Verification and repair
The checks at the end of every run, the zero-row rule, how publication works, row-drop alerts, private rows on test runs, and what an automatic repair looks like from your side.
A run that fetched pages is not yet a run that produced good data. Before anything reaches a dataset, the outputs are checked; if they fail, nothing is published and, for scheduled runs, Jason is asked to find out why. This page describes those checks, what publishes and what does not, and what you see when a repair happens.
- Run finishesevery step done
- Count outputsrows per output
- Zero-row checkfail if none
- Publishnew dataset revisions
- Notifyalerts, watches
- Healthcards update
What happens at the end of a live run
- Outputs are counted. Each declared output’s rows are saved and the run’s Rows collected is their total.
- The zero-row check runs. A live run that finished with zero output rows is turned into a failure with the error Pipeline produced zero output rows. — unless the pipeline’s zero-row policy allows empty results (below). Test and debug runs, cancelled runs and file-only pipelines are exempt.
- Outputs are published. A successful live run writes each output to its dataset as a new revision, versions any deliverable files, and attaches per-row source attribution (which is what the Sources tab and row provenance read). Test and debug runs never reach this step.
- Notifications go out. Alert rules on run status fire, and anyone who pressed the notify bell is told.
- Pipeline health is recomputed. Cards and Home update. A run that has completed but whose publish is still being finalised shows Awaiting publication for a moment.
Checks specific to the data (types, required values, duplicates, matching rules) are part of the pipeline itself, expressed as rules in the statement of work. Rows that fail such a rule are held with a reason rather than published; the rules say exactly what is held and why.
The zero-row rule
An empty result is treated as a failure by default because it is the most common sign of a broken source: the page changed, the listing moved, the site started blocking. Publishing an empty revision would silently empty every dashboard and destination downstream.
| Policy | Behaviour |
|---|---|
| Fail on zero rows (default) | The run becomes Failed with Pipeline produced zero output rows. Nothing is published; the dataset keeps its last revision. A scheduled run with this failure is eligible for automatic repair. |
| Allow empty results | The run completes and publishes an empty revision. |
The policy is part of the pipeline and is set by asking Jason. Use the second only when “nothing today” is a real answer: a feed of new filings, a list of active recalls.
This pipeline collects new planning applications. Some days there are none. Treat zero rows as a normal, successful result and publish them.
Publication and private rows
| Run | Rows go to |
|---|---|
| Live run, Resumed run | The dataset, as a new revision, when the run completes. Outputs shows Added to dataset Live product offers as v9. |
| Test run (Jason’s, while building or repairing) | The run’s own Outputs tab only, marked not published. Publication · Private · does not publish on the Summary. |
| Debug run | As a test run. |
| A Live run that was stopped | Partial rows stay on the run’s Outputs tab, not published. |
Because test runs never publish, Jason can exercise a candidate version against real pages while your dataset, dashboards and destinations keep serving the last verified revision. You can open any test run from the rail (violet left edge) and read its rows exactly as Jason saw them.
How a dataset revision is written (replace the dataset, or append with de-duplication on a key) is part of the pipeline; see Dataset versions.
Row-drop alerting
A run can complete and still be wrong: a retailer that now shows ten products instead of a thousand produces rows, just far fewer. That is not a failure the checks above catch, so it is an alert rule instead of a gate.
An alert rule on run status with the row drop condition fires when a run completed, was not capped by a small item limit, there is a stable baseline of recent full runs, and this run’s rows fell below half of the trailing median. The alert carries the rows, the baseline median and how many runs formed it. It can be limited to scheduled runs.
The rule does not hold or unpublish anything; by the time it fires the rows are published. Its job is to get a person to look. See Alert rules.
Rule: Live product offers run status · row drop · scheduled runs only · Slack #market-radar. Alert: “Live run #41 completed with 612 rows; the median of the last 7 full runs was 48,210.”
Independent review before activation
Every change to a pipeline — the first build, a change you asked for, a restore, a comment commit, a manual repair, an automatic repair — follows one path. Jason produces a candidate version, runs test runs against it, and hands it to an independent reviewer that checks the output against the statement of work. Only a pass activates the candidate. A fail, or a request for more evidence, sends it back to Jason; the current version keeps running meanwhile.
From your side: the version chip does not move until the review passes; test runs appear in the rail; then a new version appears with a build, edit, Auto-heal or Manual heal badge and a changelog line, and the statement of work and steps update. See Versions and changelog and Agents, workers and repairs.
Automatic repair
When a run fails, transient problems (a timeout, a rate limit) are retried by the workers first. A run that still fails is a candidate for Auto Heal.
When a repair starts
All of these must hold:
A zero-row failure under the default policy is an ordinary failure and qualifies.
What you see
- The failure. Home’s Needs attention shows the pipeline with Failure 12 minutes ago · Last run failed; its card turns red; the run’s head is red with the error on Logs.
- Jason starts work. The repair runs on its own, without a chat you have to follow. If you open the pipeline in a chat meanwhile, Jason knows a repair is already in progress and says so.
- Test runs. Jason reads the logs and activity, refetches the failing page, changes the pipeline, and runs it. Each attempt is a violet Test run in the runs rail, with its rows on its own Outputs tab and nothing published.
- A new version. When the reviewer passes the candidate, the version chip advances with a purple Auto-heal badge, and the changelog explains the fix: Handle Amazon AggregateOffer price payloads while preserving the existing Offer extraction fallback.
- The next run. The failed run is not re-run automatically. The next scheduled slot uses the new version. You can also open the failed run and press Resume to finish its work on the repaired version now.
- Repaired. Once that next live run publishes, the pipeline’s card shows Repaired with a wrench, in the brand colour, until a later run replaces it. The Needs attention entry clears.
Changelog, v2: “Amazon changed its product data from a single offer to an aggregate of offers, so the price field was empty on 184,220 rows. Accept both shapes, keeping the previous one as a fallback. Verified on 40 products from the failed run; all prices present.”
Today a repair is visible in those separate places rather than as one story. A single view of what a repair changed is on the way.
When Jason hands over to you
Repair is a recovery process, not a guarantee. Two exits lead to you. Both leave the run failed with its reason on the run, and both reach you in Slack or email if you have an alert rule for failures:
The last verified dataset revision stays available throughout, so downstream dashboards, destinations and share links keep working on known-good data.
Manual repair
Fix with Jason on a failed run starts the same candidate, test, review, activate path in a chat you can see from the start. The resulting version is badged Manual heal when Jason records it as a repair, or edit when it turned out to be a change. Turning Enable Auto Heal off never disables this.
Turn on an alert rule for run failures so escalations reach your team where they already work, and leave Enable Auto Heal on so the easy cases are fixed before anyone reads the alert.