ReddRadar
Agent skill
pipeline
Run dlt pipelines and Temporal workflows for SignalRoom. Use when syncing data sources, triggering workflows, checking pipeline status, or debugging data ingestion issues.
Install this agent skill to your Project
npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/development/pipeline
SKILL.md
Pipeline Operations
Available Sources
| Source | Description | Schedule |
|---|---|---|
everflow |
Affiliate conversions/revenue | Hourly 7am-11pm ET |
redtrack |
Ad spend tracking | Hourly 7am-11pm ET |
s3_exports |
CSV files from S3 (Sticky.io) | Daily 6am ET |
posthog |
PostHog analytics | Stubbed |
mautic |
Mautic contacts/campaigns | Stubbed |
google_sheets |
Google Sheets data | Stubbed |
API Reference: See docs/API_REFERENCE.md for live docs, auth, and request/response examples.
Run Pipeline Directly (Local Testing)
# Activate environment
source .venv/bin/activate
# Run a specific source
python scripts/run_pipeline.py everflow
python scripts/run_pipeline.py redtrack
python scripts/run_pipeline.py s3_exports
# Dry run (see what would happen)
python scripts/run_pipeline.py everflow --dry-run
Trigger via Temporal (Production)
# Trigger and return immediately
python scripts/trigger_workflow.py everflow
# Trigger and wait for completion
python scripts/trigger_workflow.py everflow -w
# Trigger with notification on success
python scripts/trigger_workflow.py everflow -w --notify
Check Pipeline Status
# View worker logs (local)
make logs-worker
# View Fly.io logs (production)
fly logs
# Temporal Cloud UI
# https://cloud.temporal.io/namespaces/signalroom-713.nzg5u/workflows
Active Schedules
View in Temporal Cloud UI or:
python scripts/setup_schedules.py --list
Current schedules:
hourly-sync-everflow-redtrack- Hourly 7am-11pm ETdaily-sync-s3- Daily 6am ET
Troubleshooting
"Unknown source" error:
Source not registered in src/signalroom/pipelines/runner.py
Credentials error:
Check .env has correct API keys
Database connection failed:
- Verify Supabase pooler settings (port 6543)
- User must be
postgres.{project_ref}
Activity timeout: Pipeline took >30 min. Filter resources or increase timeout.
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
Didn't find tool you were looking for?