Stream Processing Windowing Designer

Overview

Designs optimal windowing strategies for stream processing. This skill provides expertise in window types, watermarks, and trigger strategies for streaming applications.

Capabilities

• Window type selection (tumbling, sliding, session, global)
• Watermark strategy design
• Late data handling
• Trigger configuration
• Window aggregation optimization
• State management recommendations
• Exactly-once semantics configuration

Input Schema

{
  "useCase": "string",
  "eventTimeField": "string",
  "latencyRequirements": {
    "maxLatencyMs": "number",
    "allowedLateMs": "number"
  },
  "aggregations": ["object"]
}

Output Schema

{
  "windowConfig": {
    "type": "string",
    "size": "string",
    "slide": "string"
  },
  "watermarkConfig": "object",
  "triggerConfig": "object",
  "lateDataHandling": "object"
}

Target Processes

• Streaming Pipeline
• Feature Store Setup

Usage Guidelines

1. Define use case and event time field
2. Specify latency requirements
3. List aggregation operations needed
4. Consider late data arrival patterns

Best Practices

• Choose window type based on business requirements
• Configure watermarks based on expected lateness
• Use appropriate triggers for latency vs completeness tradeoff
• Plan state management for long windows
• Test with realistic event time distributions