Agent skill

actionbook-web-test

Run browser-based web tests against websites using Actionbook CLI. Activate when the user wants to test a website workflow, run smoke tests, verify a user flow, check if a web application works, run regression tests, or validate browser-based interactions. Supports test definition, execution, assertion, reporting, and json-ui visual report generation.

View SKILL.md on GitHub Repository
Stars 1,464
Forks 102

Install this agent skill to your Project

npx add-skill https://github.com/actionbook/actionbook/tree/main/playground/actionbook-web-test

SKILL.md

When to Use This Skill

Activate when the user:

  • Asks to "test", "verify", "check", or "validate" a website workflow
  • Wants to run smoke tests or health checks on a web application
  • Needs to verify a user flow works end-to-end (login, checkout, search, etc.)
  • Asks to "run regression tests" or "does this still work?"
  • Wants to confirm a deployment didn't break functionality
  • Needs to monitor a website's functionality on a schedule
  • Builds browser-based test suites without writing Playwright/Cypress code

What actionbook-web-test Provides

actionbook-web-test transforms web tests from coded test scripts into declarative YAML workflows executed by AI agents via Actionbook CLI.

Benefit How
AI-native recovery When a selector fails, the agent snapshots the live page and finds the equivalent element
Actionbook-managed selectors Pre-verified selectors with health scores — no manual maintenance
Cross-project reusability YAML workflows work anywhere Actionbook CLI is installed
No test framework required No Playwright/Cypress/Jest setup — just actionbook browser commands
Human-readable tests YAML workflows are readable by non-developers
Visual test reports json-ui powered HTML reports with metrics, step details, and failure screenshots

Test Workflow Format

Tests are defined as YAML files in a tests/ directory. Each file describes one test workflow.

yaml
name: example-test
description: What this test verifies
url: https://example.com
tags: [smoke, critical]
timeout: 30000  # ms, default 30000

# Pre-fetch verified selectors from Actionbook
actions:
  - "example.com:/:default"

# Environment variables (support {{env.VAR}} templates)
env:
  USERNAME: "test-user"
  PASSWORD: "{{env.TEST_PASSWORD}}"

# Browser setup options
setup:
  headless: true
  auto_dismiss_dialogs: true
  no_animations: true

# Ordered test steps
steps:
  - name: Open page
    action: open
    url: "https://example.com"

  - name: Verify loaded
    assert:
      - type: element-exists
        selector: "#main-content"

Full schema reference: workflow-format.md

Step Types

Each step has a name and either an action (browser command) or assert (verification checks).

Actions → CLI Command Mapping

Action CLI Command Required Fields
open actionbook browser open <url> url
click actionbook browser click "<selector>" selector
fill actionbook browser fill "<selector>" "value" selector, value
type actionbook browser type "<selector>" "value" selector, value
select actionbook browser select "<selector>" "value" selector, value
hover actionbook browser hover "<selector>" selector
press actionbook browser press <key> key
wait actionbook browser wait "<selector>" selector
wait-fn actionbook browser wait-fn "<expression>" expression
wait-idle actionbook browser wait-idle —
wait-nav actionbook browser wait-nav —
snapshot actionbook browser snapshot —
screenshot actionbook browser screenshot —
text actionbook browser text [selector] selector (optional)
eval actionbook browser eval "expression" expression
upload actionbook browser upload "<selector>" "<file-path>" selector, file_path
scroll actionbook browser scroll <direction> direction (up/down/top/bottom/to)
emulate actionbook browser emulate <device> device
info actionbook browser info "<selector>" selector
console actionbook browser console --level error —
close actionbook browser close —

Step Options

yaml
- name: Accept cookies if present
  action: click
  selector: "[data-testid='cookie-accept']"
  on_fail: continue     # skip | abort (default) | continue
  retry: 1              # override retry count
  timeout: 5000         # step-level timeout override
  condition: element-exists "[data-testid='cookie-banner']"

Assertion Types

Steps can include assert blocks to verify expected outcomes. Common types listed below; see assertion-types.md for the complete reference.

Type Description CLI Mapping
text-contains Element text contains string browser text "<selector>" + string check
text-equals Element text exactly matches browser text "<selector>" + exact match
text-matches Text matches regex pattern browser text "<selector>" + regex
url-contains Current URL contains string browser eval "location.href"
url-equals Current URL exactly matches browser eval "location.href"
element-exists Element present in DOM browser wait "<selector>" --timeout 5000
element-not-exists Element NOT present browser eval "!document.querySelector(...)"
element-visible Element is visible browser eval visibility check
element-hidden Element hidden or absent Inverse of element-visible
element-count Element count matches condition browser eval "querySelectorAll(...).length"
attribute-equals Element attribute matches browser eval "getAttribute(...)"
attribute-contains Attribute contains substring browser eval "getAttribute(...)"
page-title-contains Page title contains string browser eval "document.title"
eval-truthy JS expression evaluates truthy browser eval "<expression>"
console-no-errors No JS errors in console browser console --level error
network-no-failures No HTTP 4xx/5xx errors Network monitoring via CDP
screenshot-match Visual regression comparison browser screenshot + pixel diff
performance-under Performance metric under threshold browser eval performance timing

Assertion Examples

yaml
# Text assertions
- name: Verify welcome message
  assert:
    - type: text-contains
      selector: "[data-testid='welcome']"
      value: "Welcome back"

# URL assertions
- name: Verify redirect
  assert:
    - type: url-contains
      value: "/dashboard"

# Element count with operator
- name: Verify search results
  assert:
    - type: element-count
      selector: ".search-result"
      operator: ">="
      value: 5

# JS evaluation
- name: Verify cart state
  assert:
    - type: eval-truthy
      expression: "JSON.parse(localStorage.getItem('cart')).items.length > 0"

Execution Flow

Step 0: Pre-flight Checks

Before running any test, verify the environment is ready:

bash
# 1. Check browser connection
actionbook browser status
# If no browser → open one with setup flags

# 2. Verify target site is reachable (fast check, no rendering)
actionbook browser fetch <url> --format text --timeout 10000 --lite
# If fails → report site unreachable, skip all tests for this domain

# 3. Start console error monitoring
actionbook browser console --level error --duration 0 &
# Capture JS errors throughout the test session

Pre-flight failures should be reported clearly — distinguish "test failed" from "environment broken".

Step 1: Discover

Parse YAML workflow files from the tests/ directory. Filter by --filter flag (matches tags or name).

bash
# Run all tests
/actionbook-web-test run tests/

# Run smoke tests only
/actionbook-web-test run tests/smoke/

# Filter by tag
/actionbook-web-test run tests/ --filter critical

Step 2: Setup

For each workflow:

  1. Pre-fetch selectors: actionbook search + actionbook get "<action-id>" for each entry in actions
  2. Resolve template variables ({{env.VAR}}, {{timestamp}}, etc.)
  3. Restore auth state if setup.profile is specified (cookies/storage from previous session)
  4. Open browser with configured flags:
    bash
    actionbook --auto-dismiss-dialogs --no-animations browser open <url>
  5. If setup.emulate is set, apply device emulation:
    bash
    actionbook browser emulate iphone-14

Step 3: Execute

For each step in order:

  1. Check condition (if present) — skip step if condition is false
  2. Pre-check element (for interaction steps): use info to verify element state
    bash
    actionbook browser info "<selector>"
# Returns: bounding box, visibility, enabled state, attributes
  3. Translate action to actionbook browser CLI command
  4. Execute the command
  5. If step has assert block: run each assertion check
  6. On PASS: log success, continue to next step
  7. On FAIL: enter recovery (Step 4) or handle per on_fail setting
  8. After the last step of each test (regardless of PASS/FAIL/SKIP): capture a screenshot of the current page state. This screenshot will be embedded in the report under that test's section.
    bash
    # Auto-capture at end of each test — save to a per-test temp file
actionbook browser screenshot /tmp/test-<test-name>-final.png
base64 -i /tmp/test-<test-name>-final.png | tr -d '\n' > /tmp/test-<test-name>-final-b64.txt

Smart Waits: Always prefer wait-fn over eval "setTimeout":

bash
# BAD: blind delay
actionbook browser eval "new Promise(r => setTimeout(r, 800))"

# GOOD: wait for condition
actionbook browser wait-fn "document.querySelector('#sidebar').offsetWidth < 100" --timeout 5000

# GOOD: wait for element state change
actionbook browser wait-fn "document.querySelector('.loading').style.display === 'none'" --timeout 10000

# GOOD: wait for URL change after click
actionbook browser wait-fn "window.location.href.includes('/dashboard')" --timeout 10000

Step 4: Recover

Error Recovery Strategy Retries
Selector not found snapshot → find equivalent selector → retry step 1
Navigation timeout wait "<selector>" --timeout 15000 → retry (use wait instead of wait-idle in extension mode) 1
Element not clickable scroll to "<selector>" + wait → retry 1
Element not visible info "<selector>" to check state → scroll/wait → retry 1
Login wall detected Check cookies list → if no auth, pause for user to log in, resume 0 (manual)
Anti-bot / CAPTCHA Add --stealth, fingerprint rotate → retry 1
Assertion failure Screenshot + log actual vs expected (genuine failure) 0
Browser crash Re-open browser, restart from failed step 1

Selector recovery detail:

When a selector from Actionbook or the workflow YAML fails at runtime:

bash
# 1. Snapshot the live page
actionbook browser snapshot --interactive --compact --max-tokens 800

# 2. Find the equivalent element in the snapshot output
# 3. Use the new selector to retry the failed step

Step 5: Teardown

bash
# Capture any accumulated JS errors before closing
actionbook browser console --level error

# Close browser
actionbook browser close

Always close the browser, even on test failure.

Step 6: Report

Generate test results in the requested format. See Report Generation for details.

Selector Strategy

Selectors come from three sources: Actionbook API (verified, health-scored), workflow YAML (static), and live snapshot (runtime fallback).

Priority Source When to Use
1 actionbook search + get Build phase — discover and pre-fill selectors for target pages
2 data-testid / aria-label Stable attributes written directly in workflow YAML
3 CSS selector Specified directly in workflow steps
4 actionbook browser snapshot Runtime fallback when all above selectors fail

Test Construction Flow

Tests are built using Actionbook selectors, not hand-written:

bash
# 1. Search for the target page's action
actionbook search "reddit homepage sidebar navigation search" --domain reddit.com

# 2. Get the full page structure with verified selectors
actionbook get "reddit.com:/search/:default"
# → Returns page structure with inline CSS selectors:
#   Sidebar container: #left-sidebar-container
#   Collapse button: #flex-nav-collapse-button
#   Feed sort links: a[href*='/hot/?feed=home']
#   Search results: main
#   ...

# 3. Use these selectors to write the YAML test

This means you don't need to manually inspect the page — Actionbook provides verified, health-scored selectors that are regularly maintained.

Advanced Selectors

Shadow DOM

Standard CSS selectors cannot pierce Shadow DOM boundaries. To interact with elements inside a Shadow DOM, use actionbook browser eval to traverse the shadow root:

bash
# Click a button inside a Shadow DOM
actionbook browser eval "document.querySelector('host-element').shadowRoot.querySelector('button.inner').click()"

# Read text from inside a Shadow DOM
actionbook browser eval "document.querySelector('host-element').shadowRoot.querySelector('.label').textContent"

In a workflow step:

yaml
- name: Click shadow DOM button
  action: eval
  expression: "document.querySelector('host-element').shadowRoot.querySelector('button.submit').click()"

For deeply nested shadow roots, chain .shadowRoot.querySelector(...) calls.

Extension Mode Constraints

When running via the browser extension backend (as opposed to a full Playwright/CDP connection), certain features are unavailable or behave differently:

Constraint Workaround
wait-idle not supported Use wait "<selector>" with timeout, or wait-fn "<condition>" for state changes. Only use eval "new Promise(r => setTimeout(r, N))" as last resort for pure animation delays.
fill/type incompatible with Web Components Web Components with Shadow DOM inputs (e.g., Reddit's faceplate-search-input) cannot be filled via fill/type. Use eval to set .value directly, or navigate to the target URL with query parameters
Shadow DOM selector piercing Standard CSS selectors cannot reach inside Shadow DOM. Use eval with .shadowRoot.querySelector()

Example — Web Component input workaround:

yaml
# Instead of: fill "input[name='q']" "search term"
# Navigate directly to the search results URL:
- name: Navigate to search results
  action: open
  url: "https://www.reddit.com/search/?q=actionbook"

Iframes

Elements inside iframes exist in a separate document context. Use actionbook browser eval to access iframe content:

bash
# Click a button inside an iframe
actionbook browser eval "document.querySelector('iframe#payment').contentDocument.querySelector('button.pay').click()"

# Read text from inside an iframe
actionbook browser eval "document.querySelector('iframe#payment').contentDocument.querySelector('.total').textContent"

In a workflow step:

yaml
- name: Fill iframe form field
  action: eval
  expression: "document.querySelector('iframe#payment').contentDocument.querySelector('#card-number').value = '4111111111111111'"

Note: contentDocument only works for same-origin iframes. Cross-origin iframes cannot be accessed via JavaScript due to browser security policies.

Multi-Tab Handling

When an action (e.g., clicking a link with target="_blank") opens a new tab, the browser context remains on the original tab. Use these commands to manage multiple tabs:

bash
# List all open tabs
actionbook browser pages

# Switch to a specific tab by page ID
actionbook browser switch <page_id>

In a workflow:

yaml
- name: Click link that opens new tab
  action: click
  selector: "a[target='_blank']"

- name: Switch to new tab
  action: eval
  expression: "/* use 'actionbook browser pages' to find the new tab's page_id, then 'actionbook browser switch <page_id>' */"

Note: After actionbook browser pages, identify the new tab by its URL or title, then use actionbook browser switch <page_id> to move context to that tab. All subsequent commands will execute against the switched tab.

Result Reporting

Console Output (default)

actionbook-web-test results
========================
  PASS  google-search-smoke    (6 steps, 3.2s)
  FAIL  app-login-flow         (step 4: "Click submit" - selector not found)
  SKIP  checkout-e2e           (requires login)

Results: 1 passed, 1 failed, 1 skipped (3 total)
Duration: 12.4s

JSON Output (--json)

bash
/actionbook-web-test run tests/ --json --output results.json
json
{
  "timestamp": "2026-03-13T10:00:00Z",
  "results": [
    {
      "name": "google-search-smoke",
      "status": "passed",
      "steps": { "total": 6, "passed": 6, "failed": 0 },
      "assertions": { "total": 3, "passed": 3, "failed": 0 },
      "duration": 3200
    },
    {
      "name": "app-login-flow",
      "status": "failed",
      "steps": { "total": 7, "passed": 3, "failed": 1, "skipped": 3 },
      "failedStep": {
        "name": "Click submit",
        "error": "Selector not found: button[type='submit']",
        "screenshot": "screenshots/app-login-flow-step4.png"
      },
      "duration": 8100
    }
  ],
  "summary": { "passed": 1, "failed": 1, "skipped": 1, "total": 3, "duration": 12400 }
}

Report Generation

After test execution, generate a visual HTML report using json-ui. The agent constructs a json-ui JSON document from the test results, then renders it to HTML.

How It Works

  1. Collect results — Track each step's status, duration, error, and screenshot file path during execution
  2. Encode screenshots — Convert all captured PNG screenshots to base64 (store in temp files)
  3. Build json-ui JSON — Use a Python/Node script to construct the Report node tree, embedding base64 screenshots as Image components in each section
  4. Render to HTML — npx @actionbookdev/json-ui render report.json -o report.html
  5. Open in browser — Show the report to the user

json-ui Report Template

The agent should generate a JSON document following this structure:

json
{
  "type": "Report",
  "props": { "title": "Actionbook Test Report", "theme": "auto" },
  "children": [
    {
      "type": "BrandHeader",
      "props": {
        "badge": "Actionbook Test",
        "poweredBy": "actionbook-web-test",
        "showBadge": true
      }
    },
    {
      "type": "Section",
      "props": { "title": "Summary", "icon": "chart" },
      "children": [
        {
          "type": "MetricsGrid",
          "props": {
            "cols": 5,
            "metrics": [
              { "label": "Total", "value": "3", "icon": "list" },
              { "label": "Passed", "value": "1", "trend": "up", "icon": "check" },
              { "label": "Failed", "value": "1", "trend": "down", "icon": "warning" },
              { "label": "Skipped", "value": "1", "icon": "skip" },
              { "label": "Duration", "value": "12.4s", "icon": "clock" }
            ]
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "Test Results", "icon": "code" },
      "children": [
        {
          "type": "Table",
          "props": {
            "columns": [
              { "key": "status", "label": "Status" },
              { "key": "name", "label": "Test Name" },
              { "key": "steps", "label": "Steps" },
              { "key": "assertions", "label": "Assertions" },
              { "key": "duration", "label": "Duration" }
            ],
            "rows": [
              {
                "status": "PASS",
                "name": "google-search-smoke",
                "steps": "6/6",
                "assertions": "3/3",
                "duration": "3.2s"
              },
              {
                "status": "FAIL",
                "name": "app-login-flow",
                "steps": "3/7",
                "assertions": "1/2",
                "duration": "8.1s"
              }
            ],
            "striped": true
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "google-search-smoke — Step Details", "icon": "check", "collapsible": true },
      "children": [
        {
          "type": "ContributionList",
          "props": {
            "numbered": true,
            "items": [
              { "title": "Open Google", "badge": "PASS", "description": "`browser open https://google.com` (0.5s)" },
              { "title": "Verify search box", "badge": "PASS", "description": "`browser wait \"input[name='q']\"` (0.3s)" },
              { "title": "Fill search query", "badge": "PASS", "description": "`browser fill \"input[name='q']\" \"actionbook\"` (0.2s)" },
              { "title": "Submit search", "badge": "PASS", "description": "`browser press Enter` (0.1s)" },
              { "title": "Verify results loaded", "badge": "PASS", "description": "`browser wait \"#search\"` (1.5s)" },
              { "title": "Verify result count", "badge": "PASS", "description": "assert element-count >= 5 (0.6s)" }
            ]
          }
        },
        {
          "type": "Image",
          "props": {
            "src": "data:image/png;base64,...",
            "alt": "google-search-smoke — final state",
            "caption": "Page state after test completed (PASS)"
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "app-login-flow — Step Details", "icon": "warning", "collapsible": true },
      "children": [
        {
          "type": "Callout",
          "props": {
            "type": "important",
            "title": "Step 4: Click submit",
            "content": "Selector not found: `button[type='submit']`\n\nThe submit button was not found on the page. This may indicate a UI change or the element has not loaded."
          }
        },
        {
          "type": "ContributionList",
          "props": {
            "numbered": true,
            "items": [
              { "title": "Open login page", "badge": "PASS", "description": "`browser open https://app.example.com/login` (0.8s)" },
              { "title": "Fill username", "badge": "PASS", "description": "`browser fill \"#email\" \"test@example.com\"` (0.2s)" },
              { "title": "Fill password", "badge": "PASS", "description": "`browser fill \"#password\" \"***\"` (0.1s)" },
              { "title": "Click submit", "badge": "FAIL", "description": "`browser click \"button[type='submit']\"` — Selector not found" },
              { "title": "Verify redirect to dashboard", "badge": "SKIP", "description": "Skipped due to previous failure" },
              { "title": "Check welcome message", "badge": "SKIP", "description": "Skipped due to previous failure" },
              { "title": "Close browser", "badge": "SKIP", "description": "Skipped due to previous failure" }
            ]
          }
        },
        {
          "type": "Image",
          "props": {
            "src": "data:image/png;base64,...",
            "alt": "app-login-flow — failure state",
            "caption": "Page state at point of failure (step 4: Click submit)"
          }
        }
      ]
    },
    {
      "type": "BrandFooter",
      "props": {
        "timestamp": "2026-03-13T10:00:12Z",
        "attribution": "Generated by actionbook-web-test"
      }
    }
  ]
}

json-ui Component Usage Guide

Test Report Section json-ui Component Purpose
Header BrandHeader Report title, badge, branding
Summary metrics MetricsGrid Pass/fail/skip counts, total duration
Test list Table Per-test status, step counts, duration
Failure details Callout (type: important) Error message, selector, expected vs actual
Per-test screenshot Image Screenshot at end of each test (PASS or FAIL), MUST use base64 data URL. Placed after ContributionList in each test's collapsible section
Failure callout Callout (type: important) + Image For failed tests: error callout before the step list, screenshot shows failure state
Step-by-step log ContributionList Ordered steps with pass/fail badges
Console errors Callout (type: warning) JS errors captured during test
Environment info DefinitionList Browser version, viewport, URL, profile
Footer BrandFooter Timestamp, attribution

Rendering the Report

The json-ui package is @actionbookdev/json-ui on npm. Use npx to run it without global install:

bash
# 1. Agent writes test results to a JSON file (test-report.json)
# 2. Render to HTML and open in browser:
npx @actionbookdev/json-ui render test-report.json -o test-report.html

# Don't auto-open browser:
npx @actionbookdev/json-ui render test-report.json -o test-report.html --no-open

# Pipe from stdin:
cat test-report.json | npx @actionbookdev/json-ui render - -o test-report.html

IMPORTANT: Always write the JSON to a file first, then render with npx @actionbookdev/json-ui. Do NOT attempt to generate raw HTML directly — json-ui handles all styling, theming, dark mode, and responsive layout.

Embedding Screenshots in Reports

Screenshots MUST be embedded as base64 data URLs using Image components. Local file paths (file://) do NOT work — browsers block loading local files from HTML for security reasons.

Capture and encode workflow:

bash
# 1. Capture screenshot to a temp file
actionbook browser screenshot /tmp/step-screenshot.png

# 2. Encode to base64 (store in a temp file for later assembly)
base64 -i /tmp/step-screenshot.png | tr -d '\n' > /tmp/step-screenshot-b64.txt

Embed in json-ui JSON using Image component:

json
{
  "type": "Image",
  "props": {
    "src": "data:image/png;base64,<base64-encoded-content>",
    "alt": "Step description",
    "caption": "Screenshot at this step"
  }
}

IMPORTANT: Every screenshot step should produce an Image node in the report JSON. Place the Image node inside the corresponding section, after the ContributionList of step details. Use a Python/Node script to assemble the final JSON from base64 files — do NOT attempt to inline large base64 strings manually.

Report assembly pattern (recommended):

bash
# Use a Python script to build the report JSON with embedded base64:
python3 << 'PYEOF'
import json

# Load base64 data from temp files
with open("/tmp/step-screenshot-b64.txt") as f:
    b64 = f.read()

image_node = {
    "type": "Image",
    "props": {
        "src": f"data:image/png;base64,{b64}",
        "alt": "Screenshot description",
        "caption": "Caption text"
    }
}

# ... insert into report JSON children ...
PYEOF

Per-Test Detail Sections

Every test (PASS, FAIL, or SKIP) gets its own collapsible Section in the report. Each section contains a ContributionList of step details, followed by an Image with the test's final screenshot embedded as base64:

json
{
  "type": "Section",
  "props": { "title": "app-login-flow — Step Details", "icon": "code", "collapsible": true },
  "children": [
    {
      "type": "ContributionList",
      "props": {
        "numbered": true,
        "items": [
          { "title": "Open login page", "badge": "PASS", "description": "browser open https://app.example.com/login (0.8s)" },
          { "title": "Fill username", "badge": "PASS", "description": "browser fill \"#email\" \"test@example.com\" (0.2s)" },
          { "title": "Fill password", "badge": "PASS", "description": "browser fill \"#password\" \"***\" (0.1s)" },
          { "title": "Click submit", "badge": "FAIL", "description": "browser click \"button[type='submit']\" — Selector not found" },
          { "title": "Verify redirect to dashboard", "badge": "SKIP", "description": "Skipped due to previous failure" }
        ]
      }
    },
    {
      "type": "Image",
      "props": {
        "src": "data:image/png;base64,iVBORw0KGgo...",
        "alt": "app-login-flow — failure screenshot",
        "caption": "Screenshot at point of failure (step 4)"
      }
    }
  ]
}

Per-test section structure (applies to ALL tests, not just failures):

  1. Callout (type: important) — only for failed tests: error details before the step list
  2. ContributionList — step-by-step execution log with PASS/FAIL/SKIP badges
  3. Image — base64-embedded screenshot captured at the end of that test

The section icon should reflect the test outcome: "check" for PASS, "warning" for FAIL, "skip" for SKIP.

Full report format reference: report-format.md

Running Tests

bash
# Run all tests in a directory
/actionbook-web-test run tests/

# Run a single test file
/actionbook-web-test run tests/smoke/google-search.yaml

# Filter by tag
/actionbook-web-test run tests/ --filter smoke

# JSON output
/actionbook-web-test run tests/ --json --output results.json

# HTML report
/actionbook-web-test run tests/ --html --output report.html

# Verbose mode (show each CLI command)
/actionbook-web-test run tests/ --verbose

Auth State Management

Tests that require login should persist and reuse authentication state via Actionbook profiles.

Save Auth State After Login

bash
# Use a named profile — cookies and storage persist across sessions
actionbook --profile myapp browser open "https://app.example.com/login"

# After login (manual or automated), the profile saves cookies/storage automatically
# Next run with the same profile reuses the session
actionbook --profile myapp browser open "https://app.example.com/dashboard"

In Workflow YAML

yaml
setup:
  profile: "myapp-test"    # Reuse saved session

steps:
  - name: Verify already logged in
    assert:
      - type: url-contains
        value: "/dashboard"
    on_fail: continue       # If not logged in, proceed to login steps

  - name: Login if needed
    condition: url-not-contains "/dashboard"
    action: open
    url: "https://app.example.com/login"
    # ... login steps follow

Inspect Stored Auth State

bash
# Check current cookies
actionbook browser cookies list

# Check specific auth token
actionbook browser storage get "auth_token"

# Clear session to test fresh login
actionbook browser cookies clear --domain app.example.com

Login Wall Handling

When a test hits a login/auth wall:

  1. Check cookies — actionbook browser cookies list to see if session expired
  2. Pause automation — keep the browser session open
  3. Ask the user to complete login manually in the same browser window
  4. After user confirms, continue the test from where it paused
  5. If the post-login page is different, run actionbook search + actionbook get for the new page

Snapshot-First Test Generation

When the user wants to test a page but no YAML test exists, generate tests from a live page snapshot instead of writing YAML from scratch.

Auto-Generation Flow

bash
# 1. Open the target page
actionbook browser open "https://example.com/pricing"

# 2. Snapshot the interactive elements
actionbook browser snapshot --interactive --compact

# Output:
# e0 [navigation] "Main nav"
#   e1 [link] "Home" href="/"
#   e2 [link] "Pricing" href="/pricing"  [current]
#   e3 [link] "Docs" href="/docs"
# e4 [heading] "Pricing Plans"
# e5 [button] "Start Free Trial"
# e6 [button] "Contact Sales"
# e7 [region] "FAQ"
#   e8 [button] "What's included?"
#   e9 [button] "Can I cancel anytime?"

From Snapshot → YAML Test

Based on the snapshot output, generate a smoke test that verifies:

  1. Page loads: key elements from snapshot exist
  2. Navigation works: clickable links/buttons are functional
  3. Content present: headings and labels match expected text
yaml
# Auto-generated from snapshot of https://example.com/pricing
name: pricing-page-smoke
description: Verify pricing page loads with key elements and interactions
url: https://example.com/pricing
tags: [smoke, auto-generated]

steps:
  - name: Open pricing page
    action: open
    url: "https://example.com/pricing"

  - name: Verify page heading
    assert:
      - type: text-contains
        selector: "h1, h2, [role='heading']"
        value: "Pricing"

  - name: Verify CTA buttons exist
    assert:
      - type: element-exists
        selector: "button"
      - type: element-count
        selector: "button"
        operator: ">="
        value: 2

  - name: Verify navigation links
    assert:
      - type: element-exists
        selector: "a[href='/']"
      - type: element-exists
        selector: "a[href='/docs']"

  - name: Click FAQ accordion
    action: click
    selector: "[role='button']:has-text('What\\'s included?')"
    on_fail: continue

  - name: Capture final state
    action: screenshot

When to Auto-Generate

  • User says "test this page" without providing a YAML file
  • User provides a URL and says "create a smoke test"
  • A new page is added and needs basic coverage

Always show the generated YAML to the user for review before execution.

Device Emulation

Test responsive behavior by emulating mobile/tablet devices:

In Workflow YAML

yaml
setup:
  emulate: iphone-14         # Use device preset
  # Or custom viewport:
  viewport:
    width: 414
    height: 896

steps:
  - name: Verify mobile menu button visible
    assert:
      - type: element-visible
        selector: "[data-testid='mobile-menu-toggle']"

  - name: Verify desktop nav hidden on mobile
    assert:
      - type: element-hidden
        selector: "nav.desktop-nav"

Available Device Presets

Preset Resolution User Agent
iphone-14 390x844 Mobile Safari
iphone-se 375x667 Mobile Safari
pixel-7 412x915 Mobile Chrome
ipad 820x1180 Tablet Safari
desktop-hd 1920x1080 Desktop Chrome

Multi-Device Testing

Run the same test across multiple devices using matrix:

yaml
matrix:
  - { device: "iphone-14", expect_mobile: true }
  - { device: "ipad", expect_mobile: false }
  - { device: "desktop-hd", expect_mobile: false }

setup:
  emulate: "{{matrix.device}}"

steps:
  - name: Check mobile menu
    condition: eval-truthy "{{matrix.expect_mobile}}"
    assert:
      - type: element-visible
        selector: ".mobile-menu"

Console Error Monitoring

Capture JavaScript errors during test execution to catch runtime issues.

How to Use

bash
# Capture errors during a specific duration
actionbook browser console --level error --duration 5000

# Capture all levels
actionbook browser console --level all --duration 3000

In Workflow Steps

yaml
# At the end of a test, verify no JS errors occurred
- name: Verify no JavaScript errors
  assert:
    - type: console-no-errors
      ignore:
        - "favicon"
        - "analytics\\.google\\.com"
        - "third-party"

# After a specific interaction, check for errors
- name: Click submit and check for errors
  action: click
  selector: "#submit"

- name: Verify no errors after submit
  action: console
  assert:
    - type: console-no-errors

Console Error Patterns

Common patterns to ignore in console-no-errors:

yaml
ignore:
  - "favicon\\.ico"                    # Missing favicon (very common)
  - "analytics|tracking|gtag"          # Analytics scripts
  - "Failed to load resource.*\\.map"  # Source map 404s
  - "ResizeObserver loop"              # Benign browser warning
  - "third-party"                      # Third-party script errors

Security Considerations

WARNING: eval and eval-truthy execute arbitrary JavaScript in the page context.

  • eval runs any JS expression inside the browser page — it has full access to the DOM, cookies, localStorage, and any page-level APIs.
  • Never use eval with untrusted or user-supplied input. A malicious expression can exfiltrate data, modify page state, or perform actions as the logged-in user.
  • Prefer built-in assertion types (text-contains, element-exists, etc.) over eval-truthy whenever possible. Only use eval-truthy when no built-in assertion covers the check.
  • In CI, treat workflow YAML files like code — they can execute arbitrary JS via eval steps. All .test.yml files should go through code review before merging.

Screenshots and Sensitive Data

Screenshots captured during test execution may contain sensitive information (passwords, tokens, personal data visible on screen).

  • Before taking a screenshot of a page with sensitive fields, consider masking password inputs:
    yaml
    - name: Mask password field before screenshot
  action: eval
  expression: "document.querySelector('#password').value = '********'"
- name: Capture state
  action: screenshot
  • In CI, restrict artifact access permissions. Use short retention-days for artifacts containing screenshots.
  • For test flows that interact with sensitive data, use the --exclude-screenshots flag to skip automatic failure screenshots:
    bash
    actionbook test run tests/auth/ --exclude-screenshots

Log Redaction

When running in verbose mode (--verbose), CLI commands are logged including their arguments. For steps that fill password or secret fields, add sensitive: true to redact the value in logs:

yaml
- name: Fill password
  action: fill
  selector: "#password"
  value: "{{env.TEST_PASSWORD}}"
  sensitive: true   # Value will appear as "***" in verbose logs

Selector Trust Model

CSS selectors in workflow YAML files are treated as trusted input (like code). They are passed directly to browser APIs (querySelector). If selectors originate from an untrusted source (e.g., user input, external API), validate them before use to prevent injection.

References

Reference Description
workflow-format.md Complete YAML workflow schema reference
assertion-types.md All assertion types with examples
report-format.md json-ui report template and component mapping

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

actionbook/actionbook

actionbook

Activate when the user needs to interact with any website — browser automation, web scraping, screenshots, form filling, UI testing, monitoring, or building AI agents. Provides pre-verified page actions with step-by-step instructions and tested selectors.

1,464 102
Explore
actionbook/actionbook

article-exporter

Export any web article to a local Obsidian-ready Markdown directory. Fetches page content via actionbook CLI, downloads images locally, rewrites image references to relative paths, and optionally translates the article using AI. Produces a self-contained folder with README.md, images/, and an index.md navigation file.

1,464 102
Explore
actionbook/actionbook

deep-research

Deep research and analysis tool. Generates comprehensive HTML reports on any topic, domain, paper, or technology. Use when user asks to research, analyze, investigate, deep-dive, or generate a report on any subject. Supports academic papers (arXiv), technologies, trends, comparisons, and general topics.

1,464 102
Explore
actionbook/actionbook

arxiv-viewer

View, search, and download academic papers from arXiv. Supports API queries, web scraping via Actionbook, and HTML paper reading via ar5iv. Use when user asks about arxiv papers, academic papers, research papers, paper summaries, latest papers, or wants to search/download/read papers.

1,464 102
Explore
actionbook/actionbook

actionbook-scraper

Generate and verify web scraper scripts using Actionbook's verified selectors. Auto-validates generated scripts and fixes errors.

1,464 102
Explore
actionbook/actionbook

agent-browser

Automates browser interactions for form filling and web page interaction. Used by the request-website command to submit website indexing requests.

1,464 102
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results