QA Tester — Automated BeonIQ QA Skill

Automates first-pass QA testing for the BeonIQ transportation analytics dashboard. Uses Playwright MCP with Chrome to navigate the app, take screenshots, validate against acceptance criteria, and report results.

This does NOT replace human QA testing. It handles the repetitive screenshot-compare-validate loop so the QA team can focus on judgment calls and edge cases.

When to Use

Use this skill when asked to: "run QA", "test the app", "QA check", "check the deploy", "run the checklist", "test BeonIQ", or any request to validate the BeonIQ web app.

Prerequisites

• Playwright MCP must be available (browser automation)
• Always use Chrome: When launching browser, use --chrome flag / Chrome browser mode
• GitHub CLI (gh) authenticated to ChinchillaEnterprises/transportation-insight
• Slack MCP for posting results

Authentication — IMPORTANT

The BeonIQ app requires authentication. Follow this order:

1. Try Google SSO first — Look for the Google SSO login button on the login screen
2. If Okta 2FA blocks you — Do NOT try to brute force Okta. Instead, use the Google SSO login backdoor built into the login screen (bottom-right corner of the login page). This bypasses Okta 2FA entirely.
3. Never get stuck on auth — if both methods fail, report it and move on

Test Credentials

• Environment variables or session cookies from previous runs
• Dev environment: https://dev.beoniq.com (or current dev URL)
• Prod environment: https://app.beoniq.com (or current prod URL)

QA Checklist

Authentication

• Login works (Google SSO or backdoor)
• Correct company loads after login

Daily Briefing

• Content renders (no "data access error")
• Headline KPI is appropriate (not obscure metric)
• AI Summary accessible (View Full Summary link works)
• Styling matches current design (yellow glow, suggestion pills, badges)
• MoM/QoQ/YoY trend indicators present
• Data is current (not stale)

KPI Cards

• All 7 KPIs show values (not N/A unless expected)
• AI Insights (star icon) present on each KPI
• Expand/collapse behavior works
• Values change when filters applied (Mode/Direction)

Charts

• Data lines visible (all solid, correct colors)
• Date range covers expected period
• Y-axis properly truncated (not starting at 0 for large values)
• Chart responsive to KPI selection

AI Insight Section (below chart)

• AI Insight section visible below the chart/graph
• Performance Summary text present (not empty/missing)
• Key Drivers section present with bullet points
• Business Impact section present
• Recommended Actions section present
• Content is company-specific (not generic or copied from another company)
• Check EVERY company — missing on some but not others = pipeline issue
• Screenshot: {COMPANY}-03-overview-ai-insight

Outliers & Alerts

• Count matches badge number
• Modals open when clicked
• Root cause text present
• No duplicates

Recommendations

• Section visible
• Count matches badge
• No duplicate entries (same route, same bill numbers)
• Modal shows shipment details
• Type tags present (Consolidation, Cost, etc.)

Ask Data

• Suggested questions load
• At least 2 test queries return valid data
• No error states on common queries

Service Types

• MT (Managed Transport) — data loads, KPIs populate
• Brokerage — data loads, not showing MT data
• Parcel — shows correct parcel data or N/A if expected

Performance

• Page load < 3 seconds after hard refresh
• No zoom issues (stays at 100%)

Workflow

1. Launch Chrome via Playwright MCP
2. Navigate to BeonIQ login page
3. Authenticate via Google SSO (or bottom-right backdoor if Okta blocks)
4. For each service type (MT, Brokerage, Parcel):
   a. Switch to that service
   b. Take full-page screenshot
   c. Validate Daily Briefing section
   d. Validate each KPI card (count, values, AI insights)
   e. Validate Chart section
   f. Validate AI Insight section below chart (Performance Summary, Key Drivers, Business Impact, Recommended Actions)
   g. Validate Outliers section
   h. Validate Recommendations section
   i. Test Ask Data with sample question
   j. Screenshot any failures
5. Compare against GitHub issue acceptance criteria (if issue-specific QA)
6. Generate report
7. Post to Slack #internal-beoniq
8. Comment on relevant GitHub issues

Issue-Aware QA Mode

When testing a specific issue:

1. gh issue view [NUMBER] --repo ChinchillaEnterprises/transportation-insight --comments
2. Extract acceptance criteria from issue body
3. Check for linked mockup images in comments
4. Run targeted tests for that issue's scope
5. Comment pass/fail result on the issue

Report Format

Slack Report

QA Report — [Date] ([Environment])
Automated First Pass by QA Skill

PASSED (N):
- #XXX Feature description — details
- #XXX Feature description — details

FAILED (N):
- #XXX Feature description — what's wrong (screenshot attached)

NEEDS HUMAN REVIEW (N):
- Item that requires judgment call

Note: This is an automated first pass. Human QA review still needed for UX judgment, edge cases, and cross-browser testing.

GitHub Issue Comment

**Automated QA — [Date] ([Environment])**

Status: PASS / FAIL / NEEDS REVIEW

**Checked:**
- [x/✗] Item from acceptance criteria
- [x/✗] Item from acceptance criteria

**Screenshots:** [attached]

**Notes:** [any observations]

_Automated first pass — human review recommended._

GitHub Labels

Use these labels when commenting on issues:

• QA: Pass — Automated QA passed
• QA: Fail — Automated QA found issues
• QA: Regression — Visual or data regression detected
• Ready for QA — Dev complete, awaiting QA

Key URLs

• Dev: Check Slack #internal-beoniq for latest deploy URL
• Repo: ChinchillaEnterprises/transportation-insight
• Slack channel: #internal-beoniq (C09JD94HRUN)

Overnight Mode (Autonomous)

When invoked by the overnight pipeline (overnight-pipeline.sh Phase 4b), this skill runs non-interactively with no human in the loop.

Behavior Differences

• No Slack posting — the pipeline handles Slack. Just output results to stdout.
• Auto-file GitHub issues — for each FAILURE, create an issue immediately using gh issue create
• JSON output required — the LAST line of stdout must be a valid JSON summary (the pipeline parses it)
• 10-minute timeout — prioritize coverage over depth. Skip items that hang.
• Screenshots — save to resources/debug/qa-overnight/ in the transportation-insight repo

JSON Output Format

The last line printed to stdout must be:

{"date": "YYYY-MM-DD", "passed": N, "failed": N, "issues_created": [123, 124], "summary": "Brief text summary"}

GitHub Issue Template (Auto-Filed)

gh issue create \
  --repo ChinchillaEnterprises/transportation-insight \
  --title "[QA] Description of failure" \
  --label "QA: Fail" --label "bug" \
  --body "## Overnight QA Failure

**Environment:** dev
**Date:** YYYY-MM-DD
**Checklist Item:** Which item failed

### Expected
What should happen

### Actual
What happened instead

### Screenshot
Path: resources/debug/qa-overnight/FILENAME.png

_Auto-filed by overnight QA pipeline._"

Screenshot Path

Save all screenshots to:

/Users/tori/Documents/Repos/CHI/transportation-insight/resources/debug/qa-overnight/

Naming: {date}-{section}-{status}.png (e.g., 2026-03-04-kpi-cards-fail.png)

Priority Order (for timeout safety)

If running low on time, prioritize in this order:

1. Auth (if this fails, stop and report)
2. Daily Briefing
3. KPI Cards
4. Charts
5. Outliers & Recommendations
6. Ask Data
7. Service type switching
8. Performance

Sample Test Queries for Ask Data

• "What was the total cost per pound in January 2026?"
• "Which carrier has the highest cost per package?"
• "How much did I spend on 3rd party shipments?"