Agent skill
aps-doc-hist-union
Expert documentation generation for hist-union workflows. Documents historical and incremental table combination strategies, schema validation, time-based partitioning, and backfill processes. Use when documenting hist-union layers.
Install this agent skill to your Project
npx add-skill https://github.com/treasure-data/td-skills/tree/main/aps-doc-skills/hist-union
SKILL.md
APS Hist-Union Documentation Expert
Specialized skill for generating comprehensive documentation for hist-union workflows that combine historical and incremental table data.
When to Use This Skill
Use this skill when:
- Documenting hist-union workflows
- Creating documentation for historical data combination logic
- Documenting backfill processes
- Generating documentation for table merging strategies
- Documenting schema validation and consistency checks
Example requests:
"Document the hist-union workflow for events table"
"Create documentation for historical data combination"
"Document the backfill process for customer data"
"Generate hist-union documentation following this template: [Confluence URL]"
🚨 MANDATORY: Codebase Access Required
WITHOUT codebase access = NO documentation. Period.
If no codebase access provided:
I cannot create technical documentation without codebase access.
Required:
- Directory path to hist-union workflows
- Access to .dig, .sql, .yml files
Without access, I cannot extract real table names, union logic, or configurations.
Provide path: "Code is in /path/to/histunion/"
Before proceeding:
- Ask for codebase path if not provided
- Use Glob to verify files exist
- STOP if cannot read files
Documentation MUST contain:
- Real table names from actual configs
- Actual SQL UNION logic from codebase
- Real file paths and line numbers
- Working examples from actual code
NO generic placeholders. Only real, extracted data.
REQUIRED Documentation Template
Follow this EXACT structure (analyzed from production examples):
# Hist_Union Project Documentation
## Overview
{Brief description of hist-union purpose}
**Key Features:**
- {Feature from actual workflow}
- {Feature from actual workflow}
---
## Architecture
### Project Structure
{Actual directory tree from codebase}
### Configuration
**Databases:**
{Table with database names and purposes}
**Watermark Table:**
{inc_log table structure and purpose}
---
## Processing Patterns
### Pattern 1: Incremental Load (Watermark-Based)
{Explain watermark tracking using inc_log}
**SQL Pattern:**
{Real SQL showing UNION ALL with watermark logic}
### Pattern 2: Full Load (Complete Reload)
{Explain full reload pattern}
**SQL Pattern:**
{Real SQL showing full reload}
---
## Watermark Management
**inc_log Table:**
{Schema and purpose}
**Watermark Update Logic:**
{Explain how watermarks are updated}
---
## Main Workflow Configuration
**Workflow File:** {hist_union_runner.dig}
**Key Features:**
- {Feature from actual .dig file}
- {Feature from actual .dig file}
**Parallel Execution:**
{Explain _parallel settings}
---
## SQL Query Examples
### Example 1: Incremental Load Pattern
{Real SQL from codebase}
### Example 2: Full Load Pattern
{Real SQL from codebase}
### Example 3: Complex Schema
{Real SQL from codebase if applicable}
---
## Tables Processed
{Table organized by data source listing all tables}
---
## Execution Flow
{Step-by-step workflow execution}
**Execution Commands:**
{Real commands from codebase}
---
## Best Practices
{List from actual implementation}
---
## Troubleshooting
**Common Issues:**
{Real issues and solutions}
**Monitoring Queries:**
{Real SQL for checking status}
---
## Maintenance
{Maintenance tasks and schedules}
---
## Summary
{Brief recap of capabilities}
Template Usage Notes:
- Read actual workflows (.dig), SQL files, inc_log table schema
- Extract REAL watermark logic and UNION patterns
- Document REAL table names and processing patterns
- Use actual directory structure from codebase
- Include working SQL examples
- NO placeholders - only real extracted data
Summary
This skill generates production-ready hist-union documentation by:
- Reading actual .dig workflows and .sql files from codebase
- Following the exact template structure shown above
- Extracting real watermark logic, UNION patterns, and table configurations
- Creating comprehensive documentation with working SQL examples
Key capability: Transforms hist-union codebase into professional Confluence documentation with all processing patterns documented.
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
email-campaign
This skill should be used when the user asks to "create an email", "build an email campaign", "design an email template", "generate an email for a segment", "preview an email", or "push an email to Engage". Generates enterprise-grade HTML email templates with live preview in Treasure Studio and natural language editing, then pushes the final version to Treasure Engage.
action-report
YAML format reference for action reports rendered via preview_action_report. MUST be read before writing any action report YAML — defines the report structure (title, summary, actions array) and action item fields (as_is, to_be, reason, priority, category, impact) with incremental build workflow. Required by seo-analysis and any skill that produces prioritized recommendations.
grid-dashboard
YAML format reference for grid dashboards rendered via preview_grid_dashboard. MUST be read before writing any dashboard YAML — defines the page structure, 6 cell types (kpi, gauge, scores, table, chart, markdown), grid layout rules, cell merging syntax, and incremental build workflow. Required by seo-analysis and any skill that produces visual data dashboards.
seo-analysis
Runs SEO and AEO (Answer Engine Optimization) analysis on websites or specific pages. Use when the user mentions SEO, AEO, search rankings, search optimization, or wants to analyze how their pages perform in search engines and AI answers. Produces a data dashboard and action report with before/after recommendations.
aps-doc-core
Core documentation generation patterns and framework for Treasure Data pipeline layers. Provides shared templates, quality validation, testing framework, and Confluence integration used by all layer-specific documentation skills.
aps-doc-id-unification
Expert documentation generation for ID unification layers. Documents identity resolution algorithms, merge strategies, match rules, entity graphs, and multi-workflow orchestration. Use when documenting ID unification processes.
Didn't find tool you were looking for?