ReddRadar
Agent skill
release-testing
Comprehensive guide for testing Orient releases. Use this skill when asked to "test the release", "verify installation", "run release tests", "check v0.x.x", "validate installer", or when preparing a new version for release. Covers unit tests, installer E2E tests, CLI verification, integration tests, eval tests, and success criteria validation.
Install this agent skill to your Project
npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/data/release-testing
SKILL.md
Orient Release Testing
Test releases systematically before shipping.
Quick Test (5 min)
pnpm test:unit # Should pass 240+ tests
orient doctor && orient status # If installed
curl -s http://localhost:4098/health
Full Release Testing Sequence
Phase 1: Unit Tests
pnpm test:unit
Expected: ~244 passed, ~33 skipped Acceptable: Docker tests may fail if Docker isn't running (environment issue, not code bug)
Phase 2: Local Installer E2E
./installer/install-local.sh
This will:
- Check prerequisites (Node 20+, pnpm, PM2)
- Clean and rebuild all packages
- Copy to
~/.orient/ - Initialize SQLite database
- Start services via PM2
- Open dashboard
Phase 3: Verify Installation
# Load into current shell
export ORIENT_HOME="$HOME/.orient"
export PATH="$ORIENT_HOME/bin:$PATH"
# Run all checks
orient doctor # All services should show online
orient status # PM2 status table
orient version # Should show correct version
orient logs --lines 5 --nostream # Should show recent logs
# HTTP checks
curl -s http://localhost:4098/health | jq . # {"status":"ok"}
curl -s -o /dev/null -w "%{http_code}" http://localhost:4098/ # 200
curl -s -o /dev/null -w "%{http_code}" http://localhost:4098/qr/ # 200
# Database verification
ls -la ~/.orient/data/sqlite/
sqlite3 ~/.orient/data/sqlite/orient.db ".tables" # Should list 30+ tables
Phase 4: Integration Tests
./run.sh dev start
sleep 60 # Wait for services
INTEGRATION_TESTS=true pnpm test:integration
./run.sh dev stop
Expected: ~43 tests passing
Phase 5: Eval Tests
# Without API key (graceful skip)
pnpm test:eval # Should pass with skip message
# With dev server running (full eval infrastructure test)
./run.sh dev start
ANTHROPIC_API_KEY=xxx pnpm test:eval
./run.sh dev stop
Requirements:
ANTHROPIC_API_KEYenvironment variable- Dev server running (
./run.sh dev start) for OpenCode at localhost:4099
Known Limitation: The eval/OpenCode agent routing integration needs work. The eval infrastructure (loading, filtering, running) works correctly, but actual agent invocations through OpenCode may fail with session errors. Test will pass while logging a warning about this.
Success Criteria
| Check | Pass | Fail |
|---|---|---|
| Unit tests | 230+ passed | <200 passed or critical failures |
| Integration tests | 40+ passed | <35 passed |
| Services start | PM2 shows online | errored/stopped status |
| Health endpoint | {"status":"ok"} |
Connection refused or error |
| Dashboard HTTP | 200 | 4xx/5xx |
| QR endpoint HTTP | 200 | 4xx/5xx |
| Database | 30+ tables | Missing tables or errors |
orient doctor |
All green | Red status |
| CLI commands | All work | Any command fails |
Cleanup
# Stop services
orient stop
# Full uninstall
orient uninstall --force
# Or manual cleanup
pm2 delete orient
rm -rf ~/.orient
Troubleshooting
Port 4098 in use: lsof -i :4098 then kill <PID>
PM2 not starting: Check logs with pm2 logs orient --lines 50
Database errors: Delete and reinitialize:
rm ~/.orient/data/sqlite/orient.db*
cd ~/.orient/orient && pnpm --filter @orientbot/database run db:push:sqlite
Build failures: Clean rebuild:
find packages -name "dist" -type d -maxdepth 2 -exec rm -rf {} +
rm -rf .turbo node_modules/.cache
pnpm install && pnpm run build:all
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
Didn't find tool you were looking for?