ReddRadar
Agent skill
trino-to-hive-migration
Convert Trino queries to Hive when memory errors occur. Covers syntax differences (td_time_string→TD_TIME_FORMAT, REGEXP_LIKE→RLIKE, ARRAY_AGG→COLLECT_LIST) and engine selection.
Install this agent skill to your Project
npx add-skill https://github.com/treasure-data/td-skills/tree/main/sql-skills/trino-to-hive-migration
SKILL.md
Trino to Hive Migration
When to Migrate
Use Hive when Trino fails with:
Query exceeded per-node memory limitQuery exceeded distributed memory limit- Processing > 1 month of data, complex multi-way JOINs, high cardinality GROUP BY
Syntax Conversion
| Trino | Hive |
|---|---|
td_time_string(time, 'd!', 'JST') |
TD_TIME_FORMAT(time, 'yyyy-MM-dd', 'JST') |
approx_distinct(col) |
approx_distinct(col) (Hivemall - compatible!) |
approx_percentile(col, 0.95) |
percentile(col, 0.95) |
array_agg(col) |
collect_list(col) |
string_agg(col, ',') |
concat_ws(',', collect_list(col)) |
regexp_like(col, 'pattern') |
col rlike 'pattern' |
| Automatic small table join | /*+ MAPJOIN(t) */ |
Example Migration
Trino (fails with memory error):
select
td_time_string(time, 'd!', 'JST') as date,
approx_distinct(session_id) as sessions
from events
where td_time_range(time, '2024-01-01', '2024-12-31')
group by td_time_string(time, 'd!', 'JST')
Hive (works):
select
TD_TIME_FORMAT(time, 'yyyy-MM-dd', 'JST') as date,
approx_distinct(session_id) as sessions
from events
where td_time_range(time, '2024-01-01', '2024-12-31', 'JST')
group by TD_TIME_FORMAT(time, 'yyyy-MM-dd', 'JST')
Engine Selection
In Digdag:
+query_trino:
td>: queries/analysis.sql
engine: presto
+query_hive:
td>: queries/analysis.sql
engine: hive
In TD Toolbelt:
td query -d database_name -T presto "SELECT ..."
td query -d database_name -T hive "SELECT ..."
Performance Tips for Hive
-- MAPJOIN hint for small tables
select /*+ MAPJOIN(small_table) */ *
from large_table l
join small_table s on l.id = s.id
-- Process in chunks for very large datasets
where td_interval(time, '-1d', 'JST') -- Daily instead of yearly
Typical Timeline
- Trino: Fails after 5 min with memory error
- Hive: Completes in 20-30 min successfully
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
treasure-data/td-skills
email-campaign
This skill should be used when the user asks to "create an email", "build an email campaign", "design an email template", "generate an email for a segment", "preview an email", or "push an email to Engage". Generates enterprise-grade HTML email templates with live preview in Treasure Studio and natural language editing, then pushes the final version to Treasure Engage.
treasure-data/td-skills
action-report
YAML format reference for action reports rendered via preview_action_report. MUST be read before writing any action report YAML — defines the report structure (title, summary, actions array) and action item fields (as_is, to_be, reason, priority, category, impact) with incremental build workflow. Required by seo-analysis and any skill that produces prioritized recommendations.
treasure-data/td-skills
grid-dashboard
YAML format reference for grid dashboards rendered via preview_grid_dashboard. MUST be read before writing any dashboard YAML — defines the page structure, 6 cell types (kpi, gauge, scores, table, chart, markdown), grid layout rules, cell merging syntax, and incremental build workflow. Required by seo-analysis and any skill that produces visual data dashboards.
treasure-data/td-skills
seo-analysis
Runs SEO and AEO (Answer Engine Optimization) analysis on websites or specific pages. Use when the user mentions SEO, AEO, search rankings, search optimization, or wants to analyze how their pages perform in search engines and AI answers. Produces a data dashboard and action report with before/after recommendations.
treasure-data/td-skills
aps-doc-core
Core documentation generation patterns and framework for Treasure Data pipeline layers. Provides shared templates, quality validation, testing framework, and Confluence integration used by all layer-specific documentation skills.
treasure-data/td-skills
aps-doc-id-unification
Expert documentation generation for ID unification layers. Documents identity resolution algorithms, merge strategies, match rules, entity graphs, and multi-workflow orchestration. Use when documenting ID unification processes.
Didn't find tool you were looking for?