Hacker News Posts Scraper is a BrowserAct Bot template for collecting structured story data from public Hacker News pages. Configure the page type and result count, then run the Bot to return clean story records for trend monitoring, research, content discovery, developer community analysis, or automation workflows.

What Does BrowserAct Hacker News Posts Scraper Do?

This template opens a public Hacker News listing, reads each story row and its metadata row, follows the More link when needed, and returns one structured record per story.

Key Features

Extract public Hacker News stories without requiring a login.
Capture rank, title, story URL, Hacker News item URL, source domain, points, author, comment count, age, timestamp, and page type.
Configure base URL, page type, and count.
Support listing pages such as news, newest, front, ask, show, jobs, and specific item?id= pages.
Follow the public More link until the requested count is reached.

What Data Can I Extract from Hacker News?

Field	Description
`rank`	Story position on the listing page.
`title`	Story title shown by Hacker News.
`story_url`	External story URL or Hacker News item URL for self posts.
`item_url`	Hacker News discussion item URL.
`source_domain`	Source domain when Hacker News displays one.
`points`	Story points when shown.
`author`	Submitter username when shown.
`author_url`	Hacker News user profile URL.
`comment_count`	Number of comments, with `discuss` treated as zero.
`age`	Relative age shown on the page.
`published_time`	Timestamp extracted from the story metadata.
`page_type`	Hacker News page type used for the run.

How to Use Hacker News Posts Scraper

Click Run task.
Choose the Hacker News page type.
Set how many story records to collect.
Run the Bot and review the structured story results.
Reuse the Bot with another page type or connect the output to your workflow.

Input Parameters

Parameter	Required	Default	Description
`base_url`	Yes	`https://news.ycombinator.com`	Base Hacker News URL without a trailing slash.
`page_type`	Yes	`news`	Listing to open, such as `news`, `newest`, `front`, `ask`, `show`, `jobs`, or `item?id=49363433`.
`count`	Yes	`30`	Maximum number of story records to return.

Example Output

{
  "stories": [
    {
      "rank": 1,
      "title": "Example Hacker News Story",
      "story_url": "https://example.com/article",
      "item_url": "https://news.ycombinator.com/item?id=12345678",
      "source_domain": "example.com",
      "points": 243,
      "author": "example_user",
      "author_url": "https://news.ycombinator.com/user?id=example_user",
      "comment_count": 156,
      "age": "1 hour ago",
      "published_time": "2026-08-19T16:07:14",
      "page_type": "news"
    }
  ]
}

How Does it Work?

How Hacker News Posts Scraper runs

BrowserAct constructs a public Hacker News URL from base_url and page_type, then opens the listing page in a live browser. The Bot reads each story row and its following metadata row to pair the title and URL with points, author, comment count, age, and timestamp.

For listing pages, BrowserAct follows the public More link and continues collecting records until count is reached or no more public results are available. For item?id= pages, the Bot returns the single submission record and stops. Fields that Hacker News does not display for a page type, such as points on jobs pages or domains on self posts, are returned as empty values instead of blocking the run.

How to Build a New Scraper Bot with Agent Built

Step 1: Describe the Data You Need

Open BrowserAct and start Agent Built from the Home prompt box or from Create -> Build with Agent. Paste a clear request that includes the website, the records to collect, the filters or search conditions, the fields to return, the result limit, and the inputs you want to reuse later.

Agent Built prompt templates

Step 2: Let BrowserAct Build and Test the Bot

BrowserAct explores the live website, works out the navigation and extraction path, and validates the result. If BrowserAct asks for clarification, reply in the same build conversation with the missing URL, condition, field, example, or expected output.

Agent Built exploration

Step 3: Run the Bot and Review the Result

When the build is complete, run the Bot with the default inputs first. Review the structured records, source URLs, missing fields, and row count before reusing the Bot with new inputs or connecting it to another workflow.

Agent Built run inputs

Agent Built run results

Why Scrape Hacker News Posts?

Hacker News story data helps teams track developer conversations, discover fast-moving technical topics, monitor startup and product launches, collect source URLs for research, and route high-signal stories into dashboards or agents.

Who Can Use This Template?

Developer relations teams monitoring technical conversations.
Startup researchers tracking product and market signals.
Content teams discovering engineering topics.
Analysts following high-engagement links and comments.
Automation builders sending HN records into databases, alerts, reports, or agents.

How Many Results Can You Scrape?

Use count to control the number of stories returned. Listing pages show a fixed number of stories per page, so higher counts require following the More link and may take longer.

Automation and Export

Automation and export flow

BrowserAct returns structured Hacker News story records first. You can then use the output through API JSON responses, n8n workflows, Make scenarios, Zapier automations, MCP-based agent workflows configured from the Bot's Integrations tab, or manual CSV review.

Need Help?

Contact us at

Discord: Discord Community

E-mail: service@browseract.com

Hacker News Posts Scraper

History

FAQ About the template

Your next scraper starts here.