--- title: "Track the Hacker News front page" description: "Build a small pipeline against a live public page, run it, read the rows, and see where each value came from. About ten minutes." url: "https://www.jsonify.com/docs/tutorials/hacker-news" --- [Documentation](https://www.jsonify.com/docs/) Filter pages ## Getting started * [What is Jsonify?](https://www.jsonify.com/docs/) * [How pipelines work](https://www.jsonify.com/docs/how-jsonify-works) * [Your first pipeline](https://www.jsonify.com/docs/getting-started/first-pipeline) * [Workspaces](https://www.jsonify.com/docs/getting-started/workspaces) * [Glossary](https://www.jsonify.com/docs/getting-started/glossary) ## Tutorials * [Track the Hacker News front page](https://www.jsonify.com/docs/tutorials/hacker-news) * [Monitor prices across retailers](https://www.jsonify.com/docs/tutorials/price-monitoring) * [Get a Slack alert when a price drops](https://www.jsonify.com/docs/tutorials/slack-price-alert) * [Use a dataset from your agent](https://www.jsonify.com/docs/tutorials/agent-query) ## Jason * [Chatting with Jason](https://www.jsonify.com/docs/jason/chat) * [Changing a pipeline](https://www.jsonify.com/docs/jason/changing-a-pipeline) * [Agents, workers and repairs](https://www.jsonify.com/docs/data-agents) ## Pipelines * [The pipeline page](https://www.jsonify.com/docs/pipelines/overview) * [Statement of work and steps](https://www.jsonify.com/docs/pipelines/statement-of-work) * [Inputs and parameters](https://www.jsonify.com/docs/pipelines/inputs) * [Settings reference](https://www.jsonify.com/docs/pipelines/settings) * [Versions and changelog](https://www.jsonify.com/docs/pipelines/versions) ## Runs * [Running a pipeline](https://www.jsonify.com/docs/runs/running) * [Scheduling](https://www.jsonify.com/docs/runs/scheduling) * [Run detail](https://www.jsonify.com/docs/runs/run-detail) * [Run lifecycle](https://www.jsonify.com/docs/runs/lifecycle) * [Verification and repair](https://www.jsonify.com/docs/runs/verification-and-repair) ## Datasets * [Dataset pages](https://www.jsonify.com/docs/datasets/overview) * [Versions](https://www.jsonify.com/docs/datasets/versions) * [Query and export](https://www.jsonify.com/docs/datasets/query-and-export) * [Row detail and provenance](https://www.jsonify.com/docs/datasets/provenance) * [Sharing](https://www.jsonify.com/docs/datasets/sharing) ## Analytics * [Dashboards](https://www.jsonify.com/docs/analytics/dashboards) * [Newsletters](https://www.jsonify.com/docs/analytics/newsletters) ## Alerts * [Alert rules](https://www.jsonify.com/docs/alerts/rules) * [Alert history and notifications](https://www.jsonify.com/docs/alerts/history) ## Integrations * [Integrations overview](https://www.jsonify.com/docs/integrations/overview) * [Slack and email](https://www.jsonify.com/docs/integrations/slack-and-email) * [Sheets, warehouses and webhooks](https://www.jsonify.com/docs/integrations/destinations) * [Ingest and serve API](https://www.jsonify.com/docs/integrations/api) * [Connect your agent (MCP)](https://www.jsonify.com/docs/integrations/mcp) * [Command line](https://www.jsonify.com/docs/integrations/cli) ## Workspace and account * [Workspace settings](https://www.jsonify.com/docs/workspace/settings) * [Team](https://www.jsonify.com/docs/workspace/team) * [Billing and usage](https://www.jsonify.com/docs/workspace/billing) * [Data retention](https://www.jsonify.com/docs/workspace/data-retention) ## Plans * [Radar or Benchmark? Enterprise](https://www.jsonify.com/docs/radar-vs-benchmark) * [Row pricing and managed service](https://www.jsonify.com/docs/pricing-model) ## More * [Connect your agent](https://www.jsonify.com/connect) * [Pricing](https://www.jsonify.com/pricing) * [Contact support](https://www.jsonify.com/contact) [Docs](https://www.jsonify.com/docs/)/Tutorials # Track the Hacker News front page Build a small pipeline against a live public page, run it, read the rows, and see where each value came from. About ten minutes. This tutorial builds a pipeline that collects the newest submissions on Hacker News: title, link, discussion link, author, points, comments and the time it was posted. It is deliberately small so you can see every part of the product once. The screenshots are from a real run. You need A Jsonify account Rows used 90 of your free 100 Time About 10 minutes Result A dataset that grows on every run Describe the goal, review Jason's proposal, let Jason build, check the first run, then the pipeline runs on its schedule. You can refine the brief after seeing the first rows. What happens in this tutorial. ## 1\. Describe the job 1. **Open Jsonify and start a workspace.** Go to [factory.jsonify.com/setup](https://factory.jsonify.com/setup). If you already have workspaces, choose **New workspace** from the workspace list. 2. **Type the brief.** Say what to collect, from where, and what a row is. You do not need to describe how the page works. > Collect the newest public Hacker News submissions from news.ycombinator.com/newest. For each post keep the title, the link it points to, the Hacker News discussion link, the author, points, comment count and when it was posted. One row per discussion link. Start with the latest 90 posts. ![The setup dialog with the Hacker News brief typed into the goal field](/assets/images/product/tut-hn-brief.png) The setup dialog. Attach files here if you have a URL list or an example of the output you want. 1. **Continue.** Jason reads the page, proposes the fields and asks anything it is unsure about. For this brief it will ask nothing; press **Build**. ## 2\. Watch the build Jason opens the newest page, works out how posts are laid out and how the “More” link paginates, writes the pipeline, runs it against a sample, and sends it for review. You can follow along in the chat; each line of activity is something Jason did. ![Jason's task-completed card at the end of the build with links to the pipeline, dataset and first run](/assets/images/product/tut-hn-build.png) The end of the build. The card links to what Jason made. > A build ends with a real first run, so the dataset you look at next is not a sample. It is the first version of your data. ## 3\. Read the pipeline Open the pipeline. The **How it works** tab shows the statement of work: what is in scope, what is not, the parameter Jason chose (`posts = 90`), and the output contract listing every field with its type. ![Pipeline page for Latest Hacker News Posts showing the statement of work](/assets/images/product/tut-hn-pipeline.png) The statement of work is the contract for this pipeline. Ask Jason to change it and a new version is produced. Two things worth noticing: * **“Not included” is explicit.** The pipeline does not read article bodies or comment threads. If you want them, that is a change request, not an assumption. * **The parameter is yours.** `posts` controls how many submissions a run collects. Change it in **Inputs** or per run. See [Inputs and parameters](https://www.jsonify.com/docs/pipelines/inputs). ## 4\. Look at the run Open **Runs**. The first run completed in a few seconds, fetched a handful of pages and collected 90 rows across the steps. ![Run summary showing status, version, duration, pages fetched and rows collected](/assets/images/product/tut-hn-run.png) The run summary. Outputs lists the dataset it published; Logs shows what the workers did. ## 5\. Read the rows Open the dataset from the run’s outputs or from the pipeline’s **Latest outputs** tab. ![Dataset table of Hacker News posts with title, url, discussion\_url, author, points, comments and posted\_at columns](/assets/images/product/tut-hn-dataset.png) Ninety rows, one per discussion link. Columns carry formats, so URLs are links and numbers are numbers. Click any row. **Data** shows the values; **Provenance** shows the run, step and fetch that produced them, with a screenshot of the page as it was collected. Every row links to the run that produced it, the step inside that run, the fetch of the source page, and the captured page itself. Every value can be traced back to the page it came from. See [Row detail and provenance](https://www.jsonify.com/docs/datasets/provenance) for what each tab shows. ## 6\. Make it run on its own The build left the schedule off. Ask Jason, or set it yourself in **Settings**: > Run this every hour and keep appending new posts. Skip posts we already have. Jason sets the schedule and confirms that the pipeline already de-duplicates by discussion link, so re-runs only add new rows. From now on the dataset grows on its own, and each run is a new [version](https://www.jsonify.com/docs/datasets/versions) you can compare. ## What to try next [**Ask a question of the data** "Which domains were posted most this week?" in plain language, or SQL. ](https://www.jsonify.com/docs/datasets/query-and-export)[**Get an alert** Fire a Slack message when a post passes 100 points. ](https://www.jsonify.com/docs/tutorials/slack-price-alert)[**Use it from your agent** Read this dataset from Claude Code or Codex over MCP. ](https://www.jsonify.com/docs/tutorials/agent-query)[**Change the pipeline** Add the domain of each link as its own column. ](https://www.jsonify.com/docs/jason/changing-a-pipeline) [PreviousGlossary](https://www.jsonify.com/docs/getting-started/glossary) [NextMonitor prices across retailers](https://www.jsonify.com/docs/tutorials/price-monitoring) Something missing or wrong? [Tell us](https://www.jsonify.com/contact).[Build it →](https://factory.jsonify.com/setup) ## On this page