How to Clean Scraped Data Without Losing the Original Values

To clean scraped data, keep an original copy, define a rule for each field, create separate cleaned values, and compare the result with the source. Start with a small sample. A currency symbol, blank cell, or repeated URL can look like a simple formatting problem while changing the meaning of your analysis.

Inspect the problem before choosing a cleaning rule

Separate extraction problems from formatting problems. A price that includes £ can be parsed once its currency and number format are known. A field that actually contains delivery time or working hours needs a corrected extraction target. Removing its letters would produce a tidy but misleading number.

Also confirm the record structure. If a CSV contains one row with an entire array of products inside a cell, fix the table's row definition first. The website-to-Excel tutorial shows a verified one-record-per-row export. Cleaning values inside the wrong record structure leaves the original usability problem in place.

Write down what each field means before editing it. For a price, that includes the amount, currency, unit, and whether tax is included when the source states it. For a date, it includes the date convention and the precision actually available. For an identifier, it includes whether leading zeros are meaningful.

Keep raw and cleaned fields together

This guide uses Books to Scrape, a public educational sandbox. The site explicitly says its prices and ratings are fictional. Our example demonstrates data handling, not real product pricing.

The Bot keeps raw_price and raw_availability, then adds price_amount, currency, and availability. The first pair represents displayed source text; the second contains narrowly defined derived values. Title, product URL, and rating complete the eight-field record.

Build the BrowserAct cleaning example

Use a small listing first. The prompt requests both original and cleaned fields so you can inspect the transformation before connecting the output to another tool.

1. Open BrowserAct Dashboard

Click the left-side +, choose Quick Start, and enter the sandbox URL. Add the requirements below to the data input. You can also use the Custom Bot entry for the full request.

The actual BrowserAct Quick Start workspace used to create this example.

2. Copy the complete prompt

Keep the row definition, original fields, and derived-field rules together. If you adapt the target website, change the cleaning rules only after checking that site's actual values.

Build a reusable, read-only Bot for https://books.toscrape.com/.
Inputs: start_url, default https://books.toscrape.com/; max_results, default 5, integer 1-20.
Read only the supplied listing page in source order; do not follow Next or crawl the whole site.
One record must represent one book. Extract the complete card title (use the link tooltip), absolute product_url, raw_price including the pound symbol, raw_availability as rendered, and rating as an integer 1-5.
Keep raw fields. Derive price_amount by parsing this site's GBP price, currency as GBP, and availability by trimming surrounding whitespace only. No currency conversion, inferred values, or sentiment labels.
Deduplicate by absolute product_url before applying max_results. Preserve first occurrence order. Use null for missing or unparseable fields.
Return one books collection of flat records, with no extra metadata or serialized nested results cell. The native CSV download must contain one book per row with these eight columns.
Do not click Add to basket, submit forms, purchase, log in automatically, or change accounts. If login, CAPTCHA, 2FA, or restricted access appears, stop with a clear message for manual help.
Test five homepage books and allow independent runs with a changed limit and listing URL. Verify the actual CSV and JSON downloads.

3. Handle access requirements if they appear

The sandbox did not require login in our tests. On an authorized site that does, complete login, CAPTCHA, 2FA, or confirmation manually when requested. A stopped unattended run may require a rerun after access is resolved. Do not treat an access failure as a dataset full of missing values.

4. Review the transformation and export

Run the built Bot, compare original and cleaned columns, check the record count, and download CSV or JSON using Download result file. Keep the original download. Confirm that each numerical price matches its raw string and each availability value differs only by the intended whitespace cleanup.

Original price, rendered availability, rating, and derived numerical amount from the actual three-book Travel run. This is a crop of the real result view; the native file retains all eight fields.

What the actual output showed

The build test returned five homepage books. Independent runs of the published version then returned two homepage books and three books from the Travel category. On October 21, 2026, we checked the downloaded files against those source pages.

For the Travel example, £45.17, £49.43, and £48.87 became numerical amounts 45.17, 49.43, and 48.87. The original strings remained available. The source's rendered availability contained a leading space; the cleaned value was In stock. Full titles and product links matched the source.

The homepage's two-record CSV and JSON downloads also agreed on the number of books. The JSON used one books collection with numerical amount and rating values; the CSV contained one book per row. No manual flattening or replacement dataset was needed.

Our runs contained no repeated product URLs. We verified that the exported URLs were unique; we did not inject duplicate cards into the live source to prove every possible collision case. The practical checks below still matter when you adapt the workflow.

Handle duplicates, missing values, and dates deliberately

For duplicates, decide whether a row represents an entity or an observation. Keeping the first occurrence of a product URL works for this single-page list. A price-history dataset may need the same URL at different collection times, so deleting every repeated URL would remove useful observations.

For missing values, distinguish an absent source value from a parsing or access failure. Keep a missing amount empty or null according to your output contract. Do not replace it with zero unless that value is actually stated or a documented downstream rule requires it. The data cleaning explainer covers the difference.

For dates, keep the original string until its convention is known. 04/05/2026 is ambiguous without context. A month-only stay date does not justify inventing a day. Record the available precision, and store a full date only when the source supports one. Dates were not part of the tested book dataset; a date conversion needs its own source and output checks.

If you process the file in Excel, pandas, n8n, or another tool afterward, describe that as a separate downstream stage. Do not attribute those external transformations to the original BrowserAct run.

Check exceptions before repeating the workflow

Keep a small validation record with the requested count, returned records, unique identifiers, failed conversions, and representative source comparisons. A zero-error report is useful only if the rules actually test the fields that matter.

For a price amount, compare it with the raw string and retain the currency. For a text field, check that cleanup did not remove meaningful characters or rewrite the original words. For repeated jobs, watch for changes in missing-field frequency or source format rather than assuming yesterday's rules still fit.

This example reads one listing page and handles a known GBP format. Extending it to multiple pages, currencies, or websites requires new tests. Preserve that boundary when deciding whether the output is ready for a spreadsheet or an automated business workflow.