Verification and repair
The checks at the end of every run, the zero-row rule, how publication works, row-drop alerts, private rows on test runs, degraded pipelines, Autoheal now, and what an automatic repair looks like from your side.
A run that fetched pages is not yet a run that produced good data. Before anything reaches a dataset, the outputs are checked; if they fail, nothing is published, the pipeline is marked degraded, and Jason is asked to find out why. This page describes those checks, what publishes and what does not, and what you see when a repair happens.
- Run finishesevery step done
- Count outputsrows per output
- Zero-row checkfail if none
- Publishnew dataset revisions
- Notifyalerts, watches
- Healthcards update
What happens at the end of a live run
- Outputs are counted. Each declared output’s rows are saved and the run’s Rows collected is their total.
- The zero-row check runs. A live run that finished with zero output rows is turned into a failure with the error Pipeline produced zero output rows. — unless the pipeline’s zero-row policy allows empty results (below). Test and debug runs, cancelled runs and file-only pipelines are exempt.
- Outputs are published. A successful live run writes each output to its dataset as a new revision, versions any deliverable files, and attaches per-row source attribution (which is what the Sources tab and row provenance read). Test and debug runs never reach this step.
- Notifications go out. Alert rules on run status fire, and anyone who pressed the notify bell is told.
- Pipeline health is recomputed. Cards and Home update. A run that has completed but whose publish is still being finalised shows Awaiting publication for a moment. A failed live run makes the pipeline degraded (below).
Checks specific to the data (types, required values, duplicates, matching rules) are part of the pipeline itself, expressed as rules in the statement of work. Rows that fail such a rule are held with a reason rather than published; the rules say exactly what is held and why.
The zero-row rule
An empty result is treated as a failure by default because it is the most common sign of a broken source: the page changed, the listing moved, the site started blocking. Publishing an empty revision would silently empty every dashboard and destination downstream.
| Policy | Behaviour |
|---|---|
| Fail on zero rows (default) | The run becomes Failed with Pipeline produced zero output rows. Nothing is published; the dataset keeps its last revision. Like any failed live run, it makes the pipeline degraded and qualifies for automatic repair. |
| Allow empty results | The run completes and publishes an empty revision. |
The policy is part of the pipeline and is set by asking Jason. Use the second only when “nothing today” is a real answer: a feed of new filings, a list of active recalls.
This pipeline collects new planning applications. Some days there are none. Treat zero rows as a normal, successful result and publish them.
Publication and private rows
| Run | Rows go to |
|---|---|
| Live run, Resumed run | The dataset, as a new revision, when the run completes. Outputs shows Added to dataset Live product offers as v9. |
| Test run (Jason’s, while building or repairing) | The run’s own Outputs tab only, marked not published. Publication · Private · does not publish on the Summary. |
| Debug run | As a test run. |
| A Live run that was stopped | Partial rows stay on the run’s Outputs tab, not published. |
Because test runs never publish, Jason can exercise a candidate version against real pages while your dataset, dashboards and destinations keep serving the last verified revision. You can open any test run from the rail (violet left edge) and read its rows exactly as Jason saw them.
How a dataset revision is written (replace the dataset, or append with de-duplication on a key) is part of the pipeline; see Dataset versions.
Row-drop alerting
A run can complete and still be wrong: a retailer that now shows ten products instead of a thousand produces rows, just far fewer. That is not a failure the checks above catch, so it is an alert rule instead of a gate.
An alert rule on run status with the row drop condition fires when a run completed, was not capped by a small item limit, there is a stable baseline of recent full runs, and this run’s rows fell below half of the trailing median. The alert carries the rows, the baseline median and how many runs formed it. It can be limited to scheduled runs.
The rule does not hold or unpublish anything; by the time it fires the rows are published. Its job is to get a person to look. See Alert rules.
Rule: Live product offers run status · row drop · scheduled runs only · Slack #market-radar. Alert: “Live run #41 completed with 612 rows; the median of the last 7 full runs was 48,210.”
Independent review before activation
Every change to a pipeline — the first build, a change you asked for, a comment commit, a manual repair, an automatic repair — follows one path. (Restoring an earlier version is the exception: it republishes a version that already passed this path.) Jason produces a candidate version, runs test runs against it, and hands it to an independent reviewer that checks the output against the statement of work. Only a pass activates the candidate. A fail, or a request for more evidence, sends it back to Jason; the current version keeps running meanwhile.
From your side: the changelog does not gain a version until the review passes; test runs appear in the rail; in the chat, a Pipeline reviewer card shows the verdict (Passed, Failed, Needs evidence) with View report, which lists the reviewer’s findings and, on a pass, Activated as pipeline version 3.; then a new version appears in the changelog with a line saying what changed, and the statement of work and steps update. See Versions and changelog and Agents, workers and repairs.
Automatic repair
When a run fails, transient problems (a timeout, a rate limit) are retried by the workers first. A live run that still fails makes the pipeline degraded, and a degraded pipeline is a candidate for Auto Heal.
Degraded pipelines
A pipeline is degraded while its latest live run is a failure, whoever started it. While it is degraded:
The pipeline stops being degraded when a live run of the current version succeeds. After a repair, Jason runs the repaired version to confirm it; the Changelog then shows Recovery confirmed. The page-level details are in Degraded pipelines.
When a repair starts
Jason is asked to repair a degraded pipeline when all of these hold:
How quickly that happens depends on what started the failed run:
| Failed live run started by | When Jason is asked |
|---|---|
| The schedule or a trigger | Straight away. |
| You (Run Pipeline, a card, Latest runs), the command line, or Jason | About a minute later, when the pipeline is next checked. |
| Any of them, after an attempt that did not fix it | 24 hours after that attempt finished, at the next scheduled slot if that is sooner, or at once with Autoheal now. |
A zero-row failure under the default policy is an ordinary failure and qualifies.
Autoheal now
On a degraded pipeline the Changelog starts with a Degraded card and, when Enable Auto Heal is on and no repair is running, an Autoheal now button. It asks Jason to start a repair immediately, skipping the 24-hour wait between attempts. It reads Starting… while the request goes in. If Jason is busy with this pipeline, or a run of it is active, the toast reads Could not start autoheal; try again when that work has finished.
Autoheal now is hidden while a repair is running, when Enable Auto Heal is off, and on a pipeline marked Permanently broken. With the switch off, use Fix with Jason on the failed run instead.
What you see
- The failure. Home’s Needs attention shows the pipeline with Failure 12 minutes ago · Last run failed; its card turns amber with the degraded sentence; the pipeline page shows the degraded banner, and the Summary of the failed run names the step that failed. Select that step in Steps to read the error.
- Jason starts work. The repair runs on its own, in a chat started for it (Automatic pipeline repair). The card and banner read Automatic healing is in progress., the card turns blue, and the Changelog shows an Autoheal row with Jason working. If you open the pipeline in a chat meanwhile, Jason knows a repair is already in progress and says so.
- Test runs. Jason reads the run and its activity, refetches the failing page, changes the pipeline, and runs it. Each attempt is a violet Test run in the runs rail, with its rows on its own Outputs tab and nothing published.
- A new version. When the reviewer passes the candidate, the Changelog gains a version whose line explains the fix: Handle Amazon AggregateOffer price payloads while preserving the existing Offer extraction fallback. The Autoheal row reads Awaiting recovery.
- Recovery. Jason runs the new version live to confirm the fix. When that run publishes, the Autoheal row reads Recovery confirmed, the degraded banner goes away and Run Pipeline is enabled again. The failed run itself is not re-run; press Resume on it if you want its remaining work finished on the repaired version.
- Repaired. The pipeline’s card shows Repaired with a wrench, in the brand colour, until a later run replaces it. The Needs attention entry clears.
Changelog, v2: “Amazon changed its product data from a single offer to an aggregate of offers, so the price field was empty on 184,220 rows. Accept both shapes, keeping the previous one as a fallback. Verified on 40 products from the failed run; all prices present.”
The Changelog records each repair in one place: the Autoheal row with its outcome, and its details window with Triggering failure (the failed run), Repair conversation (Jason’s chat) and View v<n> (the version it produced). See Versions and changelog.
When Jason hands over to you
Repair is a recovery process, not a guarantee. Two exits lead to you. Both leave the pipeline degraded, both are recorded as an Autoheal row in the Changelog, and both reach you in Slack or email if you have an alert rule for failures:
The last verified dataset revision stays available throughout, so downstream dashboards, destinations and share links keep working on known-good data.
Manual repair
Fix with Jason on a failed run starts the same candidate, test, review, activate path in a chat you can see from the start. The resulting version’s changelog line says what was repaired. Turning Enable Auto Heal off never disables this, and it is the way to heal a degraded pipeline when the switch is off.
Turn on an alert rule for run failures so escalations reach your team where they already work, and leave Enable Auto Heal on so the easy cases are fixed before anyone reads the alert.