Agent skill

video-prompt-engineering

Optimize prompts for AI video generation platforms including Sora, Runway, Pika, and Kling

Stars 514
Forks 31

Install this agent skill to your Project

npx add-skill https://github.com/a5c-ai/babysitter/tree/main/library/specializations/domains/social-sciences-humanities/arts-culture/film-tv-production/skills/video-prompt-engineering

SKILL.md

Video Prompt Engineering Skill

Purpose

Create optimized prompts for AI video generation platforms that produce cinematic, production-quality footage. These prompts must communicate action, camera movement, timing, and style in a format that translates across different AI platforms.

Universal Prompt Structure

[SCENE SETUP] + [CHARACTER/SUBJECT] + [ACTION SEQUENCE] +
[CAMERA MOVEMENT] + [LIGHTING/ATMOSPHERE] + [STYLE/AESTHETIC]

Component Details

Component Content Example
Scene Setup Location, time, environment "A rain-soaked Tokyo street at night"
Subject Who/what appears "A woman in a red coat"
Action What happens (sequential) "walks forward, stops, turns to look back"
Camera Movement and framing "slow tracking shot, eye level"
Lighting Light sources, mood "neon signs reflecting on wet pavement"
Style Visual aesthetic "cinematic, blade runner aesthetic"

Platform-Specific Optimization

Sora (OpenAI)

Strengths:

  • Complex scenes
  • Multiple subjects
  • Consistent physics
  • Long duration

Prompt Style:

Natural language, paragraph format.
Describe the scene as if telling a story.
Include subtle details about atmosphere.
Mention specific camera movements by name.

Example:
"A close-up tracking shot follows a single snowflake as it falls
through the air, passing snow-covered pine branches and eventually
landing on a red mitten. The camera holds on the crystalline
structure of the snowflake for a beat before it begins to melt.
Soft, diffused winter light. Shallow depth of field with gentle
bokeh in the background."

Runway Gen-3

Strengths:

  • Motion control
  • Style transfer
  • Consistent aesthetics
  • Subject coherence

Prompt Style:

Structured, detailed prompts.
Specify motion types explicitly.
Reference camera movements precisely.
Include duration indicators.

Example:
"Cinematic shot, slow motion. A detective in a trench coat walks
through a crowded train station. Camera dollies backward maintaining
medium shot. People blur past in the foreground. Harsh overhead
lighting creates deep shadows. 1940s noir aesthetic. 4 seconds."

Pika

Strengths:

  • Quick generation
  • Image-to-video
  • Style consistency
  • Character animation

Prompt Style:

Concise, focused prompts.
One primary action per prompt.
Strong style keywords.
Clear motion direction.

Example:
"Close-up of woman's face, wind blowing through hair,
looking off camera left, soft golden hour lighting,
cinematic film grain, subtle movement"

Kling

Strengths:

  • Longer duration
  • Complex action
  • Multiple subjects
  • Realistic motion

Prompt Style:

Detailed action sequences.
Step-by-step motion description.
Clear spatial relationships.
Timing indications.

Example:
"A chef in a professional kitchen. Wide shot. He tosses
vegetables in a wok, flames rise dramatically (2 sec),
plates the dish with precise movements (3 sec),
wipes his brow and smiles at camera (2 sec).
Warm kitchen lighting, steam rising, professional quality."

Camera Movement Vocabulary

Static Shots

locked off, tripod, stable, still camera

Movement Types

- PAN: horizontal pivot (pan left, pan right)
- TILT: vertical pivot (tilt up, tilt down)
- DOLLY: camera moves (dolly in, dolly out, dolly alongside)
- TRACKING: follows subject (tracking shot, follow shot)
- CRANE: vertical lift (crane up, crane down)
- STEADICAM: smooth handheld (steadicam walk, floating camera)
- HANDHELD: naturalistic shake
- ZOOM: lens change (slow zoom, crash zoom)
- ORBIT: circles subject (360 orbit, arc shot)

Speed Modifiers

slow, gentle, smooth, quick, whip, crash, gradual

Action Description

Effective Action Verbs

walks → strides, shuffles, marches, stumbles
runs → sprints, jogs, dashes, bolts
looks → glances, stares, gazes, peers
turns → spins, pivots, rotates, wheels around
picks up → grabs, snatches, lifts, retrieves

Timing Language

slowly, gradually, suddenly, immediately,
after a beat, in one motion, over X seconds

Scene Prompt Template

markdown
## Scene [Number]: [Title]

### Setup
- **Location:** [Specific environment description]
- **Time:** [Time of day, lighting conditions]
- **Atmosphere:** [Weather, mood, ambience]

### Subject
- **Character(s):** [Who appears, wardrobe, positioning]
- **Key Props:** [Important objects in scene]

### Action Sequence
1. [First action with timing]
2. [Second action with timing]
3. [Third action with timing]

### Camera
- **Shot Type:** [Size and angle]
- **Movement:** [Specific movement description]
- **Speed:** [Movement speed]

### Technical
- **Duration:** [Total seconds]
- **Aspect Ratio:** [16:9, 2.39:1, etc.]
- **Style:** [Visual aesthetic reference]

### Platform Prompts

**Universal/Sora:**
[Paragraph-form natural language prompt]

**Runway:**
[Structured prompt with style keywords]

**Pika:**
[Concise action-focused prompt]

### Negative Prompt
[What to avoid: jittery motion, morphing, artifacts, etc.]

Style Keywords

Cinematic Quality

cinematic, film grain, anamorphic, 35mm film,
professional quality, movie scene, theatrical

Lighting

golden hour, blue hour, harsh shadows, soft light,
rim lighting, volumetric, neon glow, practical lighting

Motion

smooth motion, fluid movement, realistic physics,
natural motion, consistent speed, seamless

Atmosphere

atmospheric, moody, dramatic, serene, tense,
energetic, contemplative, mysterious

Common Issues & Solutions

Issue Solution
Subject morphing Describe subject consistently throughout
Jittery motion Add "smooth" and "fluid" keywords
Wrong timing Specify durations explicitly
Inconsistent style Use strong style anchors
Background issues Describe environment in detail
Physics problems Describe motion realistically

Quality Checklist

  • Scene environment clearly described
  • Subject/character specified in detail
  • Action sequence is logical and timed
  • Camera movement explicitly stated
  • Duration specified
  • Style/aesthetic defined
  • Platform-optimized version created
  • Negative prompts included where needed

Expand your agent's capabilities with these related and highly-rated skills.

a5c-ai/babysitter

gsd-tools

Central utility skill for GSD operations. Provides config parsing, slug generation, timestamps, path operations, and orchestrates calls to other specialized skills. Acts as the unified entry point that the original gsd-tools.cjs provided via its lib/ modules (commands, config, core, init).

514 31
Explore
a5c-ai/babysitter

model-profile-resolution

Resolve model profile (quality/balanced/budget) at orchestration start and map agents to specific models. Enables cost/quality tradeoffs by selecting appropriate AI models for each agent role.

514 31
Explore
a5c-ai/babysitter

verification-suite

Plan structure validation, phase completeness checks, reference integrity verification, and artifact existence confirmation. Provides the structured verification layer ensuring GSD artifacts are well-formed and complete.

514 31
Explore
a5c-ai/babysitter

state-management

STATE.md reading, writing, and field-level updates. Provides cross-session state persistence via .planning/STATE.md with structured fields for current task, completed phases, blockers, decisions, and quick tasks.

514 31
Explore
a5c-ai/babysitter

git-integration

Git commit patterns, formats, and conventions for GSD methodology. Provides atomic commits per task, structured commit messages, planning file commits, branch management, and milestone tag operations.

514 31
Explore
a5c-ai/babysitter

frontmatter-parsing

YAML frontmatter parsing and manipulation for .planning/ documents. Provides read, write, update, query, and validation operations on frontmatter blocks in GSD markdown artifacts.

514 31
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results