How to Track New Reddit Comments Without Duplicate Alerts

A Reddit comment monitor usually finds the same comments more than once. That is normal. The newest listing overlaps between runs. A job retries after a timeout. An edited comment reappears with the same URL. A reply moves into the collection window after its parent. If the workflow treats every collected row as new, one discussion can produce the same Slack, email, or Telegram alert repeatedly. The fix is not a narrower scan. It is durable identity. Store each Reddit commentid, compare later ru

First, separate comment duplicates from alert duplicates

There are two independent failure points:

The reverse can also happen. If the workflow marks a comment as alerted before the destination accepts it, a failed delivery may never be retried.

Treat collection state and delivery state as separate records. For production workflows, write the comment event and a pending alert to durable storage in the same transaction. A delivery worker can then retry the alert safely while enforcing a unique idempotency key.

Use comment_id as the content identity

Do not deduplicate by comment text, author, timestamp, or row number.

• Two people can post the same short reply.
• One author can post the same sentence in several places.
• A comment can be edited without becoming a new comment.
• Several comments can share the same displayed timestamp resolution.
• Result order changes as scores and reply trees change.

The BrowserAct Reddit Post Comments Scraper returns comment_id together with parent_comment_id, reply depth, author, text, score, published time, permalink, and deletion state. That gives the monitor a stable primary key while retaining the context needed to understand a reply.

Keep the permalink as a human-review link, but use comment_id for matching. Normalize the ID consistently before comparison and enforce a unique constraint on (monitor_id, comment_id) in the state store.

Store enough state to recognize change

A practical comment-state table needs more than a list of IDs:

Decide what the first run means

The first successful scan has no earlier state. Choose one of these policies before enabling notifications:

1. Baseline only: save all visible comments as baseline and send no alerts. Later runs alert only on unseen IDs. This is the safest default.
2. Lookback alert: alert only on comments newer than a defined cutoff, such as the last 30 minutes, while saving older visible comments as baseline.
3. Backfill: deliberately process existing comments as historical events. Label them as backfill so they cannot be confused with live alerts.

Do not silently classify the whole first scan as new. A popular thread with 400 existing comments can flood a notification channel before monitoring has even started.

Build a duplicate-safe Reddit comment tracker in BrowserAct

BrowserAct’s Reddit Post Comments Scraper starts from one public post URL and can collect top-level comments plus nested replies. Use it as the collection layer, then compare its structured output with the previous successful state.

1. Open BrowserAct Dashboard

Click the left-side + to create a Bot, paste the prompt into the Agent input, or open the Reddit Post Comments Scraper template and select Run task.

2. Copy the complete prompt

Replace the example URL and notification destination with your authorized workflow.

Create a reusable Bot that tracks newly discovered comments and replies on one public Reddit post without producing duplicate records or duplicate alert events.
 
Reusable inputs:
- monitor_id: reddit_post_example
- post_url: https://www.reddit.com/r/example/comments/example_id/example_post/
- previous_comment_state: optional CSV or JSON from the last successful run
- previous_delivery_state: optional CSV or JSON of alert idempotency keys
- first_run_policy: baseline_only
- scan_limit: 500
- alert_type: new_comment
 
Collection rules:
1. Open the exact public Reddit post URL.
2. Sort comments by New when the control is available.
3. Include nested replies and expand visible reply branches within the scan limit.
4. Extract actual comments only. Exclude post text, navigation, suggested posts, ads, and composer controls.
5. Overlap the previous scan deliberately. Do not stop simply because the first known comment appears; continue far enough to cover late-loaded and nested replies.
 
Return these fields for every visible comment:
- monitor_id
- post_url
- comment_id
- parent_comment_id when visible
- depth
- author when visible
- comment_text
- score when visible
- published_time when visible
- permalink
- is_deleted
- collected_at
 
Identity and status rules:
- Normalize comment_id consistently and use (monitor_id, comment_id) as the unique content key.
- Never deduplicate by text, author, score, timestamp, row position, or parent ID.
- If previous_comment_state is empty and first_run_policy is baseline_only, label every collected comment baseline.
- If comment_id is absent from the previous successful state, label it new.
- If comment_id exists and tracked fields are unchanged, label it previously_seen.
- If comment_id exists but text, score, parent, depth, or deletion state changed, label it updated and list changed_fields.
- If a previously stored comment is not visible in this limited scan, label it no_longer_visible only in the comparison report. Do not infer that it was deleted.
 
Alert rules:
- Create an alert event only for rows labeled new.
- Build alert_idempotency_key as monitor_id + ':' + comment_id + ':' + alert_type.
- If the key already exists in previous_delivery_state with status delivered, do not emit another alert.
- If the key is pending or failed, return it as retryable without creating a second event.
- Do not mark an alert delivered until the destination confirms success.
- A collection failure must not advance the successful checkpoint or replace the previous state with an empty file.
 
Return:
1. a CSV-ready current comment table;
2. JSON preserving parent_comment_id and depth;
3. a state-change table with baseline, new, updated, previously_seen, and no_longer_visible counts;
4. a pending-alert table containing one idempotency key per new comment;
5. a run summary with scanned count, unseen-ID count, updated count, delivery-retry count, reached_previous_coverage, and run status.
 
Use public or authorized pages only. Do not vote, comment, message users, join communities, or change any Reddit account. If Reddit asks for login, CAPTCHA, 2FA, age confirmation, or restricted access, pause and ask me to complete that step manually.

Keep overlap: repeated collection is expected. The stored ID set makes the overlap safe and protects against replies that appear late or outside a simple chronological boundary.

Scrape data from any website. Describe the data you need. Get a Bot—a reliable, reusable scraper. Try: “Track new comments on this Reddit post, preserve reply hierarchy, and emit one idempotent alert per unseen comment ID.” Get your Bot — Free

3. Handle login only when asked

The workflow targets public comments. If Reddit presents login, CAPTCHA, 2FA, an age gate, or restricted access, pause for authorized manual handling.

Do not ask the Bot to bypass the screen or interact with the discussion. Record access_blocked or requires_manual_action; keep the last successful state unchanged so the next valid run can compare against it.

4. Review, dedupe, and export

Run a two-pass test before connecting a live notification destination:

1. Run once with baseline_only; confirm that existing comments are stored and no alert events are created.
2. Add or wait for one authorized test comment, then run again; confirm exactly one new row and one pending idempotency key.
3. Run a third time with no new comment; confirm the same ID becomes previously_seen and produces no new alert.
4. Retry the pending delivery; confirm the destination receives one alert and the same key becomes delivered.
5. Edit the test comment; confirm the record becomes updated, retains the same comment_id, and follows the edit-alert policy.
6. Simulate a collection failure; confirm that the successful checkpoint and stored comment state do not advance.

Export the current comment table, state-change table, pending-alert table, and JSON hierarchy. Keep delivery receipts separately from the content snapshot.

The publication version should include current screenshots from a completed baseline, second scan, and forced delivery-retry test. This review draft does not invent those results.

Make overlapping scans an explicit design choice

A timestamp-only checkpoint is fragile. Several comments can share a displayed time, nested replies can load after newer top-level comments, and network or page rendering order can change between runs. Reddit’s current sorting guide includes New for comments, but the sort is a collection aid rather than a durable checkpoint.

Instead, scan a deliberate overlap and let the ID store reject known comments. The overlap can be defined by count, time, or both:

• rescan the newest 200 comments;
• rescan the last two hours plus a safety margin;
• continue until several known IDs from the last successful run have been reached;
• increase the limit when the previous coverage boundary was not reached.

The run summary should expose reached_previous_coverage. If the scan reaches its limit before reconnecting with known history, mark the run possible_gap. Do not call it fully successful simply because some comments were collected.

This is the same reason persistent monitoring tools separate scan depth from saved item limit. The system may need to inspect many familiar comments to discover a small number of unseen replies.

Preserve replies without treating parents as duplicates

A reply and its parent are different comments. Each has its own comment_id. parent_comment_id describes structure; it is not a deduplication key.

Preserve these fields together:

• comment_id for identity;
• parent_comment_id for the direct relationship;
• depth for display and analysis;
• post_url for the discussion root;
• permalink for human review.

If the parent sits outside the scan window, keep the reply and record the parent ID. Do not discard the reply because its parent row is missing. A downstream review interface can label the parent context as unavailable in the current snapshot.

Treat edits, deletions, and missing rows differently

Three states are often collapsed into one:

For text edits, consider storing text_hash, last_modified_seen_at, and a short change summary. Keep personal-data retention and internal access rules in mind; a monitoring system does not need to retain every historical version indefinitely.

Make alert delivery idempotent

An idempotency key turns retries into safe repeats. For a new-comment alert, use a deterministic key such as:

reddit_post_example:t1_abc123:new_comment

The delivery process should follow this order:

1. Save the new comment event and pending alert durably.
2. Send the alert with the same idempotency key on every retry.
3. Record the destination receipt or confirmed success.
4. Mark the key delivered.

If the destination supports its own idempotency header, pass the same key. If it does not, the sender must check its delivery table before every attempt.

Independent webhook documentation for a third-party Reddit API describes at-least-once delivery: a receiver may get the same delivery more than once and should deduplicate by a stable delivery ID. Apply the same principle to the outbound side of your Reddit monitor. A timeout is ambiguous; it does not prove that the first send failed.

Test restart and partial-failure behavior

A duplicate-safe workflow is not validated by one successful run. Test these failure cases:

When a managed monitor is a better fit

BrowserAct is useful when you want a visual, reusable collection workflow and structured comment fields. A purpose-built monitor can be a better fit when you need built-in persistent state, schedules, high-water marks, and run summaries.

For example, Apify’s current Reddit Comment Scraper documentation describes an onlyNewComments mode, stable monitoringKey, a first-run high-water mark, a separate monitoring scan limit, and scanned/new/skipped counts. Those features support the same principle: persistence belongs to the monitor, not to a temporary process memory.

Whichever tool you use, verify the behavior rather than trusting the setting name. Confirm the first-run policy, how far each run scans, what survives a restart, how edits are represented, and whether a failed run advances state.

Final checklist

Before enabling live alerts, confirm that:

• every comment is keyed by monitor_id + comment_id;
• alert delivery uses a separate deterministic idempotency key;
• the first run establishes a baseline unless backfill was requested;
• comment text is never the deduplication key;
• edits update an existing record;
• absence from a limited scan is not labeled deletion;
• overlapping scans are large enough to reconnect with known coverage;
• a failed run cannot erase or advance the last successful state;
• restart and timeout tests produce one visible alert;
• every alert links to the original Reddit permalink for human review.

Frequently asked questions

Can I deduplicate Reddit comments by permalink?

A canonical permalink can work as a secondary identifier, but comment_id is the cleaner primary key when it is available. Keep the permalink for reviewers and links.

Why not store only the last seen timestamp?

Timestamps do not fully describe ordering in nested discussions. Several comments can share a timestamp resolution, replies can appear outside a simple top-level order, and delayed rendering can change what a run sees. Use IDs plus an overlapping scan.

Should an edited comment trigger another alert?

Usually, no. Mark it updated and apply a separate edit policy. For compliance or urgent-signal monitoring, you may alert on material text changes, but use an edit-specific idempotency key such as comment_id + text_hash + edit_alert.

How long should I retain comment IDs?

At least for the lifetime of the monitor and its practical rescan horizon. If retention rules require pruning, document the cutoff and re-baseline behavior; deleting identity state can make an old comment look new again.

What if the monitor loses its state?

Stop notifications and re-establish a baseline. Do not rebuild state while sending live “new comment” alerts, because every visible comment may be misclassified as new.