Fresh Install Testing

Test Orient from a clean clone to validate the full development setup and test suite.

Triggers

• "fresh install test"
• "test from clean clone"
• "validate clean setup"
• "run full test suite on fresh clone"

Clone Location

cd /Users/tombensim/code/tombensim
git clone https://github.com/orient-bot/orient.git orient-fresh-test
cd orient-fresh-test

Note: The remote URL is https://github.com/orient-bot/orient.git (not the personal fork).

Phase 1: Environment Setup

Pre-flight Checks

# Verify LFS binaries are fetched (not pointers)
git lfs pull

# Verify bundled binary checksums match
./installer/install-local.sh --check

Run Doctor

./run.sh doctor --fix

This will:

• Validate Node.js >= 20.0.0, pnpm >= 9.0.0, Docker
• Create .env from .env.example if missing
• Create .mcp.config.local.json from template
• Run pnpm install automatically

Copy Credentials from Main Repo

cp /Users/tombensim/code/tombensim/orient/.env /Users/tombensim/code/tombensim/orient-fresh-test/.env

This copies Slack credentials and other secrets needed for live testing.

Build Packages

pnpm build:packages

Builds all 16 workspace packages. Required before running tests.

Phase 2: Dev Mode Startup

Start Dev-Local Mode (SQLite, no Docker)

./run.sh dev-local

This starts the development environment using SQLite (no PostgreSQL required).

Stop Dev Mode

./run.sh stop

Phase 3: Run Test Suite

Test Categories

Interpreting Results

Timeouts vs Failures:

• Timeout errors (120s, 90s) are usually due to OpenCode server response times under load, not functional failures
• These are acceptable in fresh install testing
• True failures will show assertion errors with specific file/line references

Expected Skips:

• Some tests are skipped by default (e.g., Slack live tests without tokens)
• Eval tests: 15 passing, ~38 failing is expected (ongoing refinement)

Phase 4: Slack Bot Live Testing

Export Credentials for Test Runner

The test runner doesn't automatically read .env. Export credentials explicitly:

cd /Users/tombensim/code/tombensim/orient-fresh-test
export SLACK_BOT_TOKEN=$(grep SLACK_BOT_TOKEN .env | cut -d= -f2)
export SLACK_USER_TOKEN=$(grep SLACK_USER_TOKEN .env | cut -d= -f2)
E2E_TESTS=true RUN_SLACK_LIVE_TESTS=true \
  SLACK_BOT_TOKEN="$SLACK_BOT_TOKEN" \
  SLACK_USER_TOKEN="$SLACK_USER_TOKEN" \
  pnpm vitest run tests/e2e/slack-live.e2e.test.ts

Note: source .env often fails due to comments or special characters in .env files.

Phase 5: WhatsApp Bot Testing

Start Dev Mode with WhatsApp

./run.sh dev-local

Scan QR Code

1. Open http://localhost:4098/qr/
2. Open WhatsApp on phone > Settings > Linked Devices > Link a Device
3. Scan the QR code

Verify Connection

curl -s http://localhost:4098/qr/status | jq .
# Should show: {"qrGenerated":true,"connected":true,"state":"open"}

Troubleshooting WhatsApp Session Conflicts

Symptom: Phone shows "logging in" but never completes, or immediately logs out.

Cause: Session conflict with another instance using the same WhatsApp account.

Solution:

1. Check logs for "loggedOut" reason:
   bashCopy

   grep -E "(loggedOut|logged out)" .dev-data/instance-0/logs/*.log
   

2. If logged out, a new QR code is automatically generated
3. Wait for "QR Code received" message in logs
4. Scan the fresh QR code

Log Locations:

• Main log: .dev-data/instance-0/logs/dashboard.log
• WhatsApp debug: .dev-data/instance-0/logs/whatsapp-*.log

Phase 6: Health Verification

API Health

curl -s http://localhost:4098/health | jq .
# Should return: {"status":"ok","timestamp":"..."}

Dashboard Accessibility

curl -s -o /dev/null -w "%{http_code}" http://localhost:4098/
# Should return: 200

WhatsApp QR Status

curl -s http://localhost:4098/qr/status | jq .
# Should return: {"qrGenerated":true,"connected":true,"state":"open"}

Database Verification

sqlite3 .dev-data/instance-0/orient.db "SELECT COUNT(*) FROM agents;"
# Should return: 5 (or more)

Phase 7: Mac Installer Testing

Method 1: Local bundled binary

./installer/install-local.sh --check   # Verify versions/checksums
./installer/install-local.sh           # Install to ~/.orient/bin/

Method 2: Full installer script

# Clean slate
rm -rf ~/.orient

# Run installer
bash installer/install.sh

# Verify
orient doctor
orient start
curl -s http://localhost:4098/health | jq .
orient stop

Method 3: Docker simulation (cleanest environment)

pnpm test:installer:docker

CLI Command Verification

After installer:

orient --help      # Shows help
orient doctor      # All checks pass
orient start       # Starts services
orient status      # Shows "online" status
orient logs        # Shows logs
orient stop        # Stops services
orient config      # Opens config in editor
orient version     # Shows 0.2.0

Cleanup

cd /Users/tombensim/code/tombensim/orient-fresh-test
./run.sh stop
cd ..
rm -rf orient-fresh-test

Quick Reference: Full Test Run

# Clone and setup
cd /Users/tombensim/code/tombensim
rm -rf orient-fresh-test
git clone https://github.com/orient-bot/orient.git orient-fresh-test
cd orient-fresh-test
git lfs pull
./run.sh doctor --fix
cp ../orient/.env .env
pnpm build:packages

# Start dev-local mode
./run.sh dev-local

# Run all tests
pnpm test:unit
INTEGRATION_TESTS=true pnpm test:integration
pnpm vitest run tests/contracts/
pnpm vitest run tests/config/
pnpm vitest run tests/services/
E2E_TESTS=true pnpm test:e2e
pnpm test:eval

# Verify health
curl -s http://localhost:4098/health | jq .

# Cleanup
./run.sh stop

Expected Results (v0.2.0 baseline)