Agent skill

image-processing

Image decoding, encoding, and manipulation using the `image` crate

View SKILL.md on GitHub Repository
Stars 21
Forks 3

Install this agent skill to your Project

npx add-skill https://github.com/johnlindquist/script-kit-next/tree/main/.opencode/skill/image-processing

SKILL.md

image-processing

The image crate provides native Rust implementations for image encoding/decoding. In script-kit-gpui, it's used for PNG encoding/decoding for app icons and clipboard images.

Crate version: 0.25 with features ["png"] only (no default features for minimal binary size)

Key Types

DynamicImage

Enum over supported buffer formats with automatic format conversion:

rust
let img = image::load_from_memory(png_data)?;        // -> DynamicImage
let rgba = img.to_rgba8();                           // -> RgbaImage (ImageBuffer<Rgba<u8>>)
let (width, height) = img.dimensions();              // GenericImageView trait

RgbaImage (ImageBuffer<Rgba, Vec>)

Fixed-format buffer for RGBA pixels:

rust
// Create from raw bytes (must be exactly width * height * 4 bytes)
let buffer = image::RgbaImage::from_raw(width, height, rgba_bytes)
    .expect("Invalid dimensions or byte count");

// Create new empty
let mut img = image::RgbaImage::new(width, height);

Frame

Animation frame wrapper used by GPUI's RenderImage:

rust
let frame = image::Frame::new(rgba_image);
let render_image = RenderImage::new(smallvec![frame]);

Pixel Types

rust
image::Rgba([255, 0, 0, 255])  // Red pixel
image::Rgb([255, 255, 255])    // White pixel (no alpha)
image::Luma([128])             // Grayscale

Usage in script-kit-gpui

PNG Decoding for App Icons (list_item.rs)

rust
pub fn decode_png_to_render_image(png_data: &[u8]) -> Result<Arc<RenderImage>, image::ImageError> {
    use image::GenericImageView;
    
    let img = image::load_from_memory(png_data)?;
    let mut rgba = img.to_rgba8();
    let (width, height) = img.dimensions();
    
    // IMPORTANT: GPUI/Metal expects BGRA format
    // Must swap R and B channels when creating RenderImage directly
    for pixel in rgba.chunks_exact_mut(4) {
        pixel.swap(0, 2); // RGBA -> BGRA
    }
    
    let buffer = image::RgbaImage::from_raw(width, height, rgba.into_raw())
        .expect("Failed to create image buffer");
    let frame = image::Frame::new(buffer);
    
    Ok(Arc::new(RenderImage::new(SmallVec::from_elem(frame, 1))))
}

PNG Encoding for Screenshots (platform.rs)

rust
use image::codecs::png::PngEncoder;
use image::ImageEncoder;

let mut png_data = Vec::new();
let encoder = PngEncoder::new(&mut png_data);
encoder.write_image(
    &final_image,           // &[u8] or ImageBuffer
    width, 
    height, 
    image::ExtendedColorType::Rgba8
)?;

Clipboard Image Handling (clipboard_history/image.rs)

rust
// Encode clipboard to PNG
let rgba_image = image::RgbaImage::from_raw(
    image.width as u32,
    image.height as u32,
    image.bytes.to_vec(),
).context("Failed to create RGBA image")?;

let mut png_data = Vec::new();
rgba_image.write_to(&mut Cursor::new(&mut png_data), image::ImageFormat::Png)?;

// Decode PNG to clipboard format
let img = image::load_from_memory_with_format(&png_bytes, image::ImageFormat::Png)?;
let rgba = img.to_rgba8();

Image Resizing for Screenshots

rust
let resized = image::imageops::resize(
    &image,
    new_width,
    new_height,
    image::imageops::FilterType::Lanczos3,  // High-quality downscaling
);

Loading Images

From File

rust
let img = image::open("path/to/image.png")?;  // Auto-detects format

From Bytes (Most Common in script-kit-gpui)

rust
// Auto-detect format
let img = image::load_from_memory(bytes)?;

// Explicit format (faster, no guessing)
let img = image::load_from_memory_with_format(bytes, image::ImageFormat::Png)?;

Dimensions Only (No Full Decode)

rust
let cursor = std::io::Cursor::new(&png_bytes);
let reader = image::ImageReader::with_format(cursor, image::ImageFormat::Png);
let (width, height) = reader.into_dimensions()?;  // Fast header-only parse

Pixel Access

Reading Pixels

rust
use image::GenericImageView;

let pixel = img.get_pixel(x, y);  // Returns Rgba<u8> or similar
let (r, g, b, a) = (pixel[0], pixel[1], pixel[2], pixel[3]);

Writing Pixels

rust
use image::GenericImage;

img.put_pixel(x, y, image::Rgba([255, 0, 0, 255]));

Iterating All Pixels

rust
// Immutable iteration
for (x, y, pixel) in img.pixels() {
    // pixel is Rgba<u8>
}

// Direct buffer access (fastest)
for pixel in rgba.chunks_exact_mut(4) {
    pixel.swap(0, 2);  // Swap R and B
}

Format Support

Features enabled in script-kit-gpui: png only

toml
image = { version = "0.25", default-features = false, features = ["png"] }

Available formats (require feature flags):

  • png - PNG decoding/encoding
  • jpeg - JPEG decoding/encoding
  • gif - GIF decoding/encoding
  • webp - WebP decoding/encoding
  • bmp, ico, tiff, etc.

Default features include many formats - disable for smaller binaries.

Memory Considerations

Large Image Safety

rust
// RgbaImage::from_raw returns None if dimensions don't match byte count
let buffer = image::RgbaImage::from_raw(width, height, bytes)
    .context("Dimension mismatch")?;

// Validate dimensions before allocation
let expected_bytes = (width as usize) * (height as usize) * 4;
if bytes.len() != expected_bytes {
    return Err(anyhow!("Invalid byte count"));
}

Avoiding Copies with SmallVec

rust
// BAD: SmallVec::from_elem clones the frame buffer
let render_image = RenderImage::new(SmallVec::from_elem(frame, 1));

// GOOD: Use smallvec! macro - no clone
use smallvec::smallvec;
let render_image = RenderImage::new(smallvec![frame]);

Decode Once, Cache Forever

rust
// WRONG: Decoding during render (called 60fps!)
fn render(&mut self, cx: &mut ViewContext<Self>) {
    let img = decode_png_to_render_image(&self.png_data);  // Slow!
}

// RIGHT: Decode once, store Arc<RenderImage>
fn new(png_data: &[u8]) -> Self {
    Self {
        cached_image: decode_png_to_render_image(png_data).ok(),
    }
}

Anti-patterns

Forgetting BGRA Conversion for Metal/GPUI

rust
// WRONG: Assumes RGBA works
let frame = image::Frame::new(rgba_image);
let render_image = RenderImage::new(smallvec![frame]);  // Colors wrong!

// RIGHT: Convert RGBA -> BGRA for Metal
for pixel in rgba.chunks_exact_mut(4) {
    pixel.swap(0, 2);
}

Not Validating Byte Length

rust
// WRONG: Panics on invalid input
let img = image::RgbaImage::from_raw(w, h, bytes).unwrap();

// RIGHT: Handle gracefully
let img = image::RgbaImage::from_raw(w, h, bytes)
    .context("Invalid dimensions or corrupt data")?;

Loading Same Image Multiple Times

rust
// WRONG: Decodes same icon for every list item
for item in items {
    let icon = decode_png(&item.icon_path);  // N decodes!
}

// RIGHT: Cache decoded images by path/hash
let icon_cache: HashMap<String, Arc<RenderImage>> = HashMap::new();

Using Default Features

rust
# WRONG: Pulls in all decoders, huge binary
image = "0.25"

# RIGHT: Only what you need
image = { version = "0.25", default-features = false, features = ["png"] }

Error Handling

All decode operations return Result<_, image::ImageError>:

rust
use image::ImageError;

match image::load_from_memory(bytes) {
    Ok(img) => // success
    Err(ImageError::Decoding(_)) => // corrupt/invalid format
    Err(ImageError::IoError(_)) => // read failure
    Err(ImageError::Limits(_)) => // image too large
    Err(e) => // other error
}

Quick Reference

Operation Code
Load PNG from bytes image::load_from_memory_with_format(bytes, ImageFormat::Png)?
Convert to RGBA img.to_rgba8()
Get dimensions img.dimensions() or (img.width(), img.height())
Create from raw RgbaImage::from_raw(w, h, bytes)?
Encode to PNG img.write_to(&mut cursor, ImageFormat::Png)?
Resize imageops::resize(&img, w, h, FilterType::Lanczos3)
Create Frame Frame::new(rgba_image)
Dimensions only ImageReader::with_format(cursor, fmt).into_dimensions()?

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

johnlindquist/script-kit-next

Generate Component Documentation

Based on existing docs styles and specific API implementations, and referencing same name stories, generate comprehensive documentation for the new component.

21 3
Explore
johnlindquist/script-kit-next

Generate Component Story

Generate a comprehensive story for a new component for as example.

21 3
Explore
johnlindquist/script-kit-next

new-component

How to write a new component of GPUI Component.

21 3
Explore
johnlindquist/script-kit-next

troubleshooting

Diagnose and fix common Script Kit issues. Use when the user reports bugs, crashes, missing features, or unexpected behavior in Script Kit GPUI.

21 3
Explore
johnlindquist/script-kit-next

script-authoring

Create and manage TypeScript scripts for Script Kit. Use when the user wants to write a new script, edit an existing script, or understand Script Kit's SDK and metadata system.

21 3
Explore
johnlindquist/script-kit-next

agents

Create mdflow-backed agent files for Script Kit. Use when the user wants to create AI agents, configure agent backends (Claude, Gemini, Codex), or manage agent metadata.

21 3
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results