Skip to main content

GitHub Copilot Browser Automation with BrowserAct

GitHub Copilot Browser Automation with BrowserAct
Introduction

GitHub Copilot can inspect a repository, change code, run tests, and prepare a pull request. Then the task reaches the live application and the agent loses the trail. A successful build does not prove that the login page renders, the authenticated dashboard loads, or the new checkout path works in a real browser. That is the job of GitHub Copilot browser automation. Install the BrowserAct Agent Skill and Copilot can use a real browser execution layer to open the deployed site, reuse an authorize

Detail
📌Key Takeaways
  1. 1GitHub Copilot supports Agent Skills in VS Code, Copilot CLI, and the Copilot cloud agent.
  2. 2BrowserAct should enter after code and API checks, when the task requires observable behavior on a live page.
  3. 3A reliable verification run needs an explicit URL, browser identity, test steps, expected states, and evidence contract.
  4. 4Use a local Chrome session for authorized logged-in testing; never paste credentials or cookies into the prompt.
  5. 5Separate read-only inspection from state-changing actions and require approval before submissions.
  6. 6Save the successful verification path as a repository Skill so future Copilot sessions use the same procedure.


The missing step after CI turns green

Imagine Copilot fixes an account-menu regression. Unit tests pass. The component test passes. The pull request is ready. There are still four questions the codebase cannot answer by itself:

  1. Did the current staging deployment receive the change?
  2. Does the page render under a real browser and authenticated account?
  3. Is the menu visible after the application finishes loading?
  4. Can a reviewer see evidence from the exact build and page state?

Traditional test automation can answer these questions when the selectors, fixtures, and environments are already maintained. The gap appears when an agent must investigate a changing interface, use an existing authorized login, adapt to the visible state, or hand an ambiguous result back to a person.

BrowserAct gives Copilot a browser-oriented execution path without turning the prompt into a long DOM dump. The agent can inspect compact indexed page state, interact with controls by index, keep browser sessions separate, and capture the result. If you are deciding between a maintained test suite and agent-driven exploration, read BrowserAct vs Playwright.

Define the finish line before opening a browser

“Check staging” is not a test. Give Copilot a small acceptance contract.

Target: https://staging.example.com/account
Build: PR #184 / commit 8a7c1d2
Browser: authorized local Chrome profile named staging-qa
Starting state: user is already signed in

Steps:
1. Open the account page.
2. Wait for the dashboard heading and account menu.
3. Open the account menu.
4. Verify that Billing, Security, and Sign out are visible.
5. Do not click Sign out and do not submit any form.

Evidence:
- final URL
- visible text for each expected item
- screenshot before and after opening the menu
- browser console errors introduced during the run
- PASS, FAIL, or BLOCKED with a short reason

This contract prevents the agent from wandering through the application and calling a page load “success.” It also creates an output that a pull-request reviewer can challenge.

Pro Tip: Include the commit, deployment ID, or release timestamp in every verification task. A perfect screenshot from the wrong deployment is still a failed test.

First gate: make Copilot discover the BrowserAct Skill

Copilot recognizes project Skills from .github/skills/, .claude/skills/, or .agents/skills/. Personal Skills can live in ~/.copilot/skills/ or ~/.agents/skills/. A repository Skill is the better default for post-deploy verification because the workflow should be reviewed and versioned with the application.

GitHub Copilot Agent Skills documentation

GitHub's Agent Skills documentation defines the discovery model used by Copilot.

The official BrowserAct repository provides this installation instruction:

Install browser-act. Skill source:
https://github.com/browser-act/skills/tree/main/browser-act
Verify it works after installation.

After installation, verify three things before asking Copilot to touch staging:

  • the Skill appears in Copilot's available Skills;
  • Copilot can run the BrowserAct CLI only after the expected approval;
  • the browser list and session state can be read without creating or deleting identities.

Do not pre-approve a third-party Skill's shell access until your team has reviewed its SKILL.md, scripts, and update path. GitHub's documentation explicitly warns that shell approval can allow scripts to execute in the working environment.

Second gate: choose the right browser identity

The browser identity is part of the test input. It should not be an accidental side effect.

Test condition

Recommended mode

Why

Public page and fresh state

Isolated or stealth browser

Clean session and reproducible starting point

Authenticated staging account

Local Chrome or fixed authorized identity

Reuse an existing login without putting secrets in the prompt

Multiple test accounts

Separate fixed identities

Prevent cookie, role, and account contamination

High-risk production action

Human-controlled session with approval

Keep the final action behind a trust boundary

For a logged-in application, connect to an authorized browser profile and identify it by a stable description such as staging-qa-admin. Do not tell the agent to search every profile for one that “looks logged in.” That widens access and makes the run hard to audit.

The full credential and session boundary is covered in How to Let AI Agents Handle Login Safely.

BrowserAct Skills

Give your agent a real browser, then turn the workflow into a Skill.

  • 1. Use browser-act when an agent needs to open, click, scroll, extract, or inspect a live site.
  • 2. Use browser-act-skill-forge when the workflow should become reusable across runs and agents.
  • 3. Keep the operational boundary simple: automate what the user can already do in the browser.

Third gate: run the workflow as an evidence-producing test

The browser task should move through observable checkpoints:

Open target URL
→ confirm hostname and deployment marker
→ confirm authenticated page shell
→ locate account-menu control
→ capture before screenshot
→ click menu
→ verify three required items
→ collect console failures
→ capture after screenshot
→ return structured verdict

Ask Copilot to stop at the first trust boundary. Reading a billing label is different from changing a subscription. Opening a settings panel is different from saving a setting. A verification Skill should encode the difference.

A useful result schema is:

{
"verdict": "PASS",
"target_url": "https://staging.example.com/account",
"build": "8a7c1d2",
"browser_identity": "staging-qa-admin",
"checks": [
{"name": "dashboard-loaded", "status": "pass"},
{"name": "account-menu-opened", "status": "pass"},
{"name": "required-items-visible", "status": "pass"}
],
"artifacts": ["before.png", "menu-open.png"],
"console_errors": [],
"notes": "No state-changing action was executed."
}

This is much more useful in a pull request than “looks good.” It can be compared across builds and consumed by another agent or CI step.

Pro Tip: Make BLOCKED a valid result. If the authorized session expired or the deployment is unavailable, the agent should report the boundary instead of improvising credentials or testing a different environment.

What happens when the session expires?

Login is where browser workflows often become unsafe. The agent sees a sign-in screen and tries to be helpful. A production workflow should do the opposite: stop, preserve the session context, and request the smallest human action required.

The recovery sequence is:

  1. Detect that the expected authenticated heading is missing.
  2. Confirm that the current hostname is allowed.
  3. Mark the check BLOCKED_AUTH, not FAIL_UI.
  4. Start a remote-assist or local handoff session.
  5. Let the user complete sign-in, 2FA, or account selection.
  6. Resume from the original page and rerun the starting-state checks.

BrowserAct's handoff pattern is described in Human-in-the-Loop Browser Automation. The important part is continuity: the human resolves the trust boundary, then Copilot continues the bounded test rather than starting an unrelated browser session.

Need a browser workflow that can pause without losing its place? Start with BrowserAct and keep human approval inside the execution path.

Turn the successful path into a repository Skill

The first run proves the path. The Skill makes it operational.

Create a focused repository Skill such as:

.github/skills/post-deploy-account-check/
├── SKILL.md
├── checks.schema.json
├── environments.md
└── report-template.md

The Skill should define:

  • when Copilot should load it;
  • allowed staging hostnames;
  • required browser identity descriptions;
  • read-only and state-changing action boundaries;
  • the ordered checkpoints;
  • PASS, FAIL, and BLOCKED meanings;
  • artifact names and output schema;
  • the recovery path for expired authentication;
  • prohibited actions such as payment, deletion, logout, and credential entry.

Keep the BrowserAct Skill responsible for browser operation and the repository Skill responsible for your application's verification contract. This separation makes the BrowserAct capability reusable while keeping project policy close to the code.

If the path becomes a repeated browser extraction or monitoring job, follow Create a Browser Automation Skill to package exploration, validation, and reruns more completely.

Rerun the test from a clean Copilot session

A workflow is not reusable because it succeeded once in a long conversation. Start a new Copilot session and provide only:

Run the post-deploy account check for PR #184 on staging.
Return the standard report and artifacts. Do not perform write actions.

The Skill should supply the rest. During the rerun, check:

  • Did Copilot select the intended Skill?
  • Did it choose the correct browser identity?
  • Did it verify the deployment before the UI?
  • Did it stop at prohibited actions?
  • Did it produce the same schema and artifact names?
  • Could a reviewer understand the verdict without reading the chat?

Track completion rate, recovery rate, P50/P90 duration, evidence completeness, and human interventions. The browser-agent reliability scorecard provides a reusable measurement method.

When to use this pattern

GitHub Copilot plus BrowserAct is a strong fit when the repository task must be verified against a deployed interface, particularly when the page is dynamic, authenticated, or not covered by maintained selectors.

Use an existing deterministic Playwright or Cypress suite when the flow is stable, assertions are already encoded, and CI needs a fast binary gate. Use an API test when the requirement is entirely at the service boundary. Use BrowserAct when the agent must understand and operate the current browser state, preserve an authorized session, gather human-reviewable evidence, or adapt to an unexpected but recoverable page.

The best workflow often combines them: unit and API tests first, deterministic end-to-end tests second, BrowserAct exploration and live verification third, and human approval at sensitive boundaries.


Agent-ready scraping

Two Skills, One Repeatable Browser Workflow

Start with live browser execution when the agent needs to understand a page. Move to Skill Forge when the same scraper should run again without re-exploring the site.

Step 1

Run once with browser-act

Give Codex, Claude Code, Cursor, Windsurf, or another agent a real browser for rendered pages, clicks, scrolling, screenshots, DOM extraction, and network inspection.

Open browser-act Skill
Step 2

Package with Skill Forge

Explore the site once, verify the extraction path, then generate a callable Skill package that other agents can reuse for batch jobs or scheduled workflows.

Open Skill Forge
Discover
Agent opens the target site and learns the working path.
Verify
Fields, pagination, limits, and failure cases are tested.
Reuse
The flow becomes a Skill that future agents can call.


Frequently Asked Questions

Can GitHub Copilot use Agent Skills?

Yes. GitHub documents Agent Skills for Copilot in VS Code, Copilot CLI, the Copilot cloud agent, code review, and other Copilot surfaces. Skill locations and available tools vary by surface, so validate the intended environment before publishing a shared workflow.

Is BrowserAct a native GitHub Copilot integration?

BrowserAct is an Agent Skill and CLI designed for agents that can load Skills and execute shell commands. Copilot supports that Skill model, but teams should still install, review, and test the BrowserAct Skill in their chosen Copilot surface rather than assuming identical behavior everywhere.

Should Copilot receive my username, password, or cookies?

No. Reuse an authorized browser session and let a human complete login or 2FA when required. Do not place credentials, session tokens, or one-time codes in prompts, repository files, or Skill documentation.

Can the workflow run against production?

Read-only production checks may be appropriate with strict hostname, identity, and action limits. State-changing production tasks should require explicit approval and a clear audit trail. Start on staging and expand scope only after repeated evidence.

Does BrowserAct replace Playwright tests?

No. Maintained deterministic tests remain valuable for stable flows. BrowserAct adds an agent-oriented browser layer for exploration, live-state verification, authenticated sessions, recovery, and human handoff.

What evidence should Copilot return?

At minimum: target and final URL, build or deployment identifier, browser identity description, checkpoint results, screenshots, relevant console failures, timestamp, and a PASS, FAIL, or BLOCKED verdict.

Stop writing automation&scrapers

Install the CLI. Run your first Skill in 30 seconds. Take action anywhere. Your agent no longer gets blocked.

Start free
free · no credit card