DEV Community Articles Search Scraper

DEV Community Articles Search Scraper is a BrowserAct Bot template for collecting structured article search records from public DEV Community and Forem-powered sites. Configure a keyword and result count, then run the Bot to return ranked article records for content research, developer marketing, trend tracking, SEO analysis, or automation workflows.

What Does BrowserAct DEV Community Articles Search Scraper Do?

This template opens DEV Community search for your keyword, reads public article search results, and returns one structured record per matching article.

Key Features

Extract public DEV Community search results without requiring a login.
Capture rank, title, article URL, author, author profile, publish date, tags, reactions, comments, and search keyword.
Configure base URL, keyword, and count.
Continue collecting results until the requested count is reached or no more public results are available.
Support DEV.to and compatible Forem community search pages.

What Data Can I Extract from DEV Community?

Field	Description
`rank`	Search result position.
`title`	Article title.
`article_url`	Public article URL.
`author_name`	Article author display name.
`author_profile_url`	Public author profile URL.
`published_date`	Publish date shown in the search result.
`tags`	Tags attached to the article.
`reaction_count`	Public reaction count when available.
`comment_count`	Public comment count when available.
`excerpt`	Article excerpt when visible. DEV search results may leave this empty.
`search_keyword`	Keyword used for the run.

How to Use DEV Community Articles Search Scraper

Click Run task.
Enter the DEV or Forem base URL.
Enter the article search keyword.
Set how many article records to collect.
Run the Bot and review the structured results.
Reuse the Bot with another keyword or connect the output to your workflow.

Input Parameters

Parameter	Required	Default	Description
`base_url`	Yes	`https://dev.to`	DEV Community or Forem base URL without a trailing slash.
`keyword`	Yes	`browser automation`	Article search keyword or phrase.
`count`	Yes	`20`	Maximum number of article records to return.

Example Output

{
  "results": [
    {
      "rank": 1,
      "title": "Browser Automation for Beginners: A Practical Guide",
      "article_url": "https://dev.to/gabby_six/browser-automation-for-beginners-a-practical-guide-158",
      "author_name": "Gabby Six",
      "author_profile_url": "https://dev.to/gabby_six",
      "published_date": "Aug 9",
      "tags": "automation, beginners, tutorial",
      "reaction_count": 0,
      "comment_count": 0,
      "excerpt": "",
      "search_keyword": "browser automation"
    }
  ]
}

How Does it Work?

How DEV Community Articles Search Scraper runs

BrowserAct constructs a public DEV search URL from base_url and keyword, opens the search result page, and collects article records from the public result data behind the visible cards. The Bot maps each result into a ranked article record with title, URL, author, profile URL, publish date, tags, reaction count, comment count, and keyword.

When count is higher than the first loaded set of results, BrowserAct continues through public search result pages until the requested count is reached or the available results are exhausted. DEV search results do not expose an article excerpt in the listing, so excerpt may be empty.

How to Build a New Scraper Bot with Agent Built

Step 1: Describe the Data You Need

Open BrowserAct and start Agent Built from the Home prompt box or from Create -> Build with Agent. Paste a clear request that includes the website, the records to collect, the filters or search conditions, the fields to return, the result limit, and the inputs you want to reuse later.

Agent Built prompt templates

Step 2: Let BrowserAct Build and Test the Bot

BrowserAct explores the live website, works out the navigation and extraction path, and validates the result. If BrowserAct asks for clarification, reply in the same build conversation with the missing URL, condition, field, example, or expected output.

Agent Built exploration

Step 3: Run the Bot and Review the Result

When the build is complete, run the Bot with the default inputs first. Review the structured records, source URLs, missing fields, and row count before reusing the Bot with new inputs or connecting it to another workflow.

Agent Built run inputs

Agent Built run results

Why Scrape DEV Community Articles?

DEV Community search data helps teams discover developer conversations, monitor topics, find technical authors, collect article references, compare engagement across tags, and route content intelligence into downstream workflows.

Who Can Use This Template?

Developer relations teams researching technical topics and authors.
Content marketers tracking article trends and engagement.
SEO teams collecting developer article references.
Product teams monitoring public discussions around tools and workflows.
Automation builders sending article records into databases, alerts, reports, or agents.

How Many Results Can You Scrape?

Use count to control the number of article records returned. Higher counts may require paging through more public search results and may stop early when the source site has no more matching records.

Automation and Export

Automation and export flow

BrowserAct returns structured DEV article records first. You can then use the output through API JSON responses, n8n workflows, Make scenarios, Zapier automations, MCP-based agent workflows configured from the Bot's Integrations tab, or manual CSV review.

Need Help?

Discord: Discord Community

E-mail: service@browseract.com

Brief