Agent skill

dd-logs

Log management - search, pipelines, archives, and cost control.

View SKILL.md on GitHub Repository
Stars 15
Forks 0

Install this agent skill to your Project

npx add-skill https://github.com/sushichan044/dotfiles/tree/main/.agents/skills/dd-logs

Metadata

Additional technical details for this skill

tags
datadog,logs,logging,search,dd-logs
globs
**/datadog*.yaml,**/*log*
author
datadog-labs
version
1.0.0
repository
https://github.com/datadog-labs/agent-skills
alwaysApply
false

SKILL.md

Datadog Logs

Search, process, and archive logs with cost awareness.

Prerequisites

Datadog Pup (dd-pup/pup) should already be installed:

bash
go install github.com/datadog-labs/pup@latest

Quick Start

bash
pup auth login

Search Logs

bash
# Basic search
pup logs search --query="status:error" --from="1h"

# With filters
pup logs search --query="service:api status:error" --from="1h" --limit 100

# JSON output
pup logs search --query="@http.status_code:>=500" --from="1h" --json

Search Syntax

Query Meaning
error Full-text search
status:error Tag equals
@http.status_code:500 Attribute equals
@http.status_code:>=400 Numeric range
service:api AND env:prod Boolean
@message:*timeout* Wildcard

Pipelines

Process logs before indexing:

bash
# List pipelines
pup logs pipelines list

# Create pipeline (JSON)
pup logs pipelines create --json @pipeline.json

Common Processors

json
{
  "name": "API Logs",
  "filter": { "query": "service:api" },
  "processors": [
    {
      "type": "grok-parser",
      "name": "Parse nginx",
      "source": "message",
      "grok": {
        "match_rules": "%{IPORHOST:client_ip} %{DATA:method} %{DATA:path} %{NUMBER:status}"
      }
    },
    {
      "type": "status-remapper",
      "name": "Set severity",
      "sources": ["level", "severity"]
    },
    {
      "type": "attribute-remapper",
      "name": "Remap user_id",
      "sources": ["user_id"],
      "target": "usr.id"
    }
  ]
}

⚠️ Exclusion Filters (Cost Control)

Index only what matters:

json
{
  "name": "Drop debug logs",
  "filter": { "query": "status:debug" },
  "is_enabled": true
}

High-Volume Exclusions

bash
# Find noisiest log sources
pup logs search --query="*" --from="1h" --json | jq 'group_by(.service) | map({service: .[0].service, count: length}) | sort_by(-.count)[:10]'
Exclude Query
Health checks @http.url:"/health" OR @http.url:"/ready"
Debug logs status:debug
Static assets @http.url:*.css OR @http.url:*.js
Heartbeats @message:*heartbeat*

Archives

Store logs cheaply for compliance:

bash
# List archives
pup logs archives list

# Archive config (S3 example)
{
  "name": "compliance-archive",
  "query": "*",
  "destination": {
    "type": "s3",
    "bucket": "my-logs-archive",
    "path": "/datadog"
  },
  "rehydration_tags": ["team:platform"]
}

Rehydrate (Restore)

bash
# Rehydrate archived logs
pup logs rehydrate create \
  --archive-id abc123 \
  --from "2024-01-01T00:00:00Z" \
  --to "2024-01-02T00:00:00Z" \
  --query "service:api status:error"

Log-Based Metrics

Create metrics from logs (cheaper than indexing):

bash
# Count errors per service
pup logs metrics create \
  --name "api.errors.count" \
  --query "service:api status:error" \
  --group-by "endpoint"

⚠️ Cardinality warning: Group by bounded values only.

Sensitive Data

Scrubbing Rules

json
{
  "type": "hash-remapper",
  "name": "Hash emails",
  "sources": ["email", "@user.email"]
}

Never Log

python
# In your app - sanitize before sending
import re

def sanitize_log(message: str) -> str:
    # Remove credit cards
    message = re.sub(r'\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b', '[REDACTED]', message)
    # Remove SSNs
    message = re.sub(r'\b\d{3}-\d{2}-\d{4}\b', '[REDACTED]', message)
    return message

Troubleshooting

Problem Fix
Logs not appearing Check agent, pipeline filters
High costs Add exclusion filters
Search slow Narrow time range, use indexes
Missing attributes Check grok parser

References/Documentation

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

sushichan044/dotfiles

terraform-style-guide

Generate Terraform HCL code following HashiCorp's official style conventions and best practices. Use when writing, reviewing, or generating Terraform configurations.

15 0
Explore
sushichan044/dotfiles

fix-github-actions-ci

GitHub Actions CI の失敗を調査して修正するためのスキルです。CI ログを分析して失敗箇所・原因を特定し、そのまま修正作業まで行います。

15 0
Explore
sushichan044/dotfiles

golang-safety

Defensive Golang coding to prevent panics, silent data corruption, and subtle runtime bugs. Use whenever writing or reviewing Go code that involves nil-prone types (pointers, interfaces, maps, slices, channels), numeric conversions, resource lifecycle (defer in loops), or defensive copying. Also triggers on questions about nil panics, append aliasing, map concurrent access, float comparison, or zero-value design.

15 0
Explore
sushichan044/dotfiles

resolve-merge-conflict

現在の branch を分岐元の最新に rebase するときに使う。まず rebase を実行し、止まった conflict を順番に解消して前に進める。長い事前調査を避けて、必要な file だけ見て片付けたいときに使う。

15 0
Explore
sushichan044/dotfiles

golang-data-structures

Golang data structures — slices (internals, capacity growth, preallocation, slices package), maps (internals, hash buckets, maps package), arrays, container/list/heap/ring, strings.Builder vs bytes.Buffer, generic collections, pointers (unsafe.Pointer, weak.Pointer), and copy semantics. Use when choosing or optimizing Go data structures, implementing generic containers, using container/ packages, unsafe or weak pointers, or questioning slice/map internals.

15 0
Explore
sushichan044/dotfiles

smart-compact

セッションのコンテキストを分析し、重要な情報を保持するためのプロンプトを生成して /compact コマンドの実行を支援するスキル。コンテキストウィンドウが逼迫してきた時や、セッションを整理したい時に使用。

15 0
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results