Row detail and provenance

Every row a pipeline publishes carries a record of how it was made. Open a row and you can see the values, the pages that were fetched, what they looked like at the time, and what was extracted from them.

Opening a row

There are two views of a row.

• The row dialog. Click any row in the dataset grid, or any card in card view. It opens over the grid with Previous row and Next row buttons (Left and Right arrow keys) that step through the rows on the current page. Escape closes it.
• The row page. Press Open in full page in the dialog, or follow a link to a row from search, an alert or a colleague. It has its own address (/datasets/{dataset}/rows/{row}).

The row dialog

The header shows the row’s name, taken from a column called title or name, with its short id in brackets. A row with neither reads “Row” and its short id. Underneath is its place in the page and dataset, for example “3 of 50 in Live product offers / main”. The dataset name links back to the dataset at that worksheet and version. On the right are Open in full page and Close.

The tabs run along the bottom in two groups:

Group	Tab	What it shows
Extracted	Data	Every field of the row as a label and value.
Extracted	JSON	The same row as JSON, for copying.
Source	Activity	How Jsonify found the row. See The Source view.
Source	The fetch’s own tabs	When the row came from exactly one page fetch, that fetch’s views sit next to Activity: Rendered and Raw for a web page, Content for other files, and for a browser session Result, Steps, Screenshots and Video. A spinner shows while they load.

The Data tab

Empty fields are hidden by default with a note such as “4 empty fields hidden” and a Show empty fields toggle. Values are rendered the same way as the grid, expanded: nested records as indented blocks, lists of records as an inner table (three rows, then Show all), images and videos as galleries.

Editing a value

In the dialog, press Edit at the bottom right of the Data tab. Each value then shows a pencil. On the row page there is no Edit button: hover a value and it reads Click to edit.

1. Click the value. It becomes a text box (a number box for numeric, currency and percentage columns).
2. Type the new value. Enter or clicking elsewhere saves; Escape cancels.
3. Check the result. A rejected value shows why, for example “abc” is not a valid number. Blank saves as empty.
4. Press Done in the dialog when you have finished.

Numbers are cleaned as you save: currency symbols, thousands separators and % are stripped, so “$1,299.00” is stored as 1299. Booleans accept true, yes or 1.

Each save writes a new dataset version with your change and makes it active, leaving the previous version intact. See Versions. Values cannot be edited in the grid itself, and rows opened from a dashboard are read-only.

Edits to a pipeline-produced dataset last until the next run publishes. If a value is wrong because the source is read wrongly, ask Jason to fix the pipeline so every future row is right.

The row page

The title is the row’s name with its short id, and the breadcrumb runs back through the dataset and version. The header has two actions:

• Copy link. Copies the page address, including the worksheet and version, to your clipboard. The label flips to Copied for a moment.
• View. A menu with Full dataset (the dataset page at this worksheet and version) and Pipeline run (the run that produced this version; disabled for uploaded and sample data).

The row page has two tabs at the bottom, Data and Source. Data works as in the dialog, except values are editable without pressing Edit. Source shows the same view as the dialog’s Activity tab. For rows without a run behind them, Source says so instead:

• Synthetic data. The row is sample data and does not correspond to any real-world source.
• Uploaded data. “This row was imported from products.csv.”

When a row has no source record at all, the page shows only Data.

The Source view

For rows a pipeline produced or an integration ingested, the Activity tab in the dialog and the Source tab on the row page show how the row was found.

From top to bottom:

• The note. “These are the actions that Jsonify took to find this data.”
• Attributed source. The website the row came from, with its favicon, and up to four link pills for URL columns on the row (source_url, url, menu_url and similar). Click a pill to open the page on the source site in a new tab. On the right, “3 unrelated batch fetches hidden.” when fetches that belonged to other rows in the same batch were left out.
• The activity list. One line per action, in the order they happened, with the time since the run started on the left and size and duration underneath. Lines include Fetch page (the address fetched), Reused page (served from the cache), Fetch attempted (a failed try), Extract data (how many rows came out), Source evidence, Web search (the query), API call, Computer use, and browser steps such as clicks and form entries.

Click a line to open its detail: the fetched page Rendered as it looked and as Raw content, the Data extracted from it, and for browser steps the result, the steps taken, screenshots and a video where one was recorded. When the row came from a single fetch, those views are already tabs in the dialog.

If the version you are viewing reused rows from an earlier run, the activity is taken from the run that produced the row. When the row’s history is long, a note reads “Showing activity within this row’s recorded lineage limit.”

Messages you may see: “No provenance data recorded for this row.” (the row was written without a run, for example by an upload into a pipeline dataset) and “No activity recorded for this row.” (a run produced it but recorded no actions for it).

From a dashboard

Some dashboard tables are built to open a row’s source. Clicking such a row opens the same row dialog on its Activity tab, with Open in full page leading to the row in its dataset. See Dashboards.

Provenance is kept for as long as the dataset version is retained. See Data retention.

Using provenance

A few things provenance is good for:

• Checking a surprising value. Open the row, go to Activity, open the fetch and look at the page as it was. Either the site really said that, or the pipeline read it wrongly. If the latter, select the row and tell Jason.
• Auditing a report. Share links include the Source view, so a reader of a shared dataset can click through to the source page for any row. See Sharing.
• Explaining a change. A column_changed alert links to the row; the row links to the fetch that observed the new value and the one before it.

This row says the 65-inch TV is $62.49, which cannot be right. What happened?

Jason reads the row’s activity, finds that the page fetch returned a listing page rather than the product page, and offers to repair the pipeline so the step follows the product link before extracting. The previous version’s value is unaffected.

Keyboard shortcuts

• Left / Right arrows in the row dialog: previous and next row on the current page.
• Escape: close the dialog, cancel an edit, close an image.
• Enter: save an edit.

What’s next