Agent skill

Voice Activity Detection (VAD)

Detect speech segments in audio using VAD tools like Silero VAD, SpeechBrain VAD, or WebRTC VAD. Use when preprocessing audio for speaker diarization, filtering silence, or segmenting audio into speech chunks. Choose Silero VAD for short segments, SpeechBrain VAD for general purpose, or WebRTC VAD for lightweight applications.

Stars 897
Forks 232

Install this agent skill to your Project

npx add-skill https://github.com/benchflow-ai/skillsbench/tree/main/tasks/speaker-diarization-subtitles/environment/skills/voice-activity-detection

SKILL.md

Voice Activity Detection (VAD)

Overview

Voice Activity Detection identifies which parts of an audio signal contain speech versus silence or background noise. This is a critical first step in speaker diarization pipelines.

When to Use

  • Preprocessing audio before speaker diarization
  • Filtering out silence and noise
  • Segmenting audio into speech chunks
  • Improving diarization accuracy by focusing on speech regions

Available VAD Tools

1. Silero VAD (Recommended for Short Segments)

Best for: Short audio segments, real-time applications, better detection of brief speech

python
import torch

# Load Silero VAD model
model, utils = torch.hub.load(
    repo_or_dir='snakers4/silero-vad',
    model='silero_vad',
    force_reload=False,
    onnx=False
)
get_speech_timestamps = utils[0]

# Run VAD
speech_timestamps = get_speech_timestamps(
    waveform[0],  # mono audio waveform
    model,
    threshold=0.6,  # speech probability threshold
    min_speech_duration_ms=350,  # minimum speech segment length
    min_silence_duration_ms=400,  # minimum silence between segments
    sampling_rate=sample_rate
)

# Convert to boundaries format
boundaries = [[ts['start'] / sample_rate, ts['end'] / sample_rate]
              for ts in speech_timestamps]

Advantages:

  • Better at detecting short speech segments
  • Lower false alarm rate
  • Optimized for real-time processing

2. SpeechBrain VAD

Best for: General-purpose VAD, longer audio files

python
from speechbrain.inference.VAD import VAD

VAD_model = VAD.from_hparams(
    source="speechbrain/vad-crdnn-libriparty",
    savedir="/tmp/speechbrain_vad"
)

# Get speech segments
boundaries = VAD_model.get_speech_segments(audio_path)

Advantages:

  • Well-tested and reliable
  • Good for longer audio files
  • Part of comprehensive SpeechBrain toolkit

3. WebRTC VAD

Best for: Lightweight applications, real-time processing

python
import webrtcvad

vad = webrtcvad.Vad(2)  # Aggressiveness: 0-3 (higher = more aggressive)

# Process audio frames (must be 10ms, 20ms, or 30ms)
is_speech = vad.is_speech(frame_bytes, sample_rate)

Advantages:

  • Very lightweight
  • Fast processing
  • Good for real-time applications

Postprocessing VAD Boundaries

After VAD, you should postprocess boundaries to:

  • Merge close segments
  • Remove very short segments
  • Smooth boundaries
python
def postprocess_boundaries(boundaries, min_dur=0.30, merge_gap=0.25):
    """
    boundaries: list of [start_sec, end_sec]
    min_dur: drop segments shorter than this (sec)
    merge_gap: merge segments if silence gap <= this (sec)
    """
    # Sort by start time
    boundaries = sorted(boundaries, key=lambda x: x[0])

    # Remove short segments
    boundaries = [(s, e) for s, e in boundaries if (e - s) >= min_dur]

    # Merge close segments
    merged = [list(boundaries[0])]
    for s, e in boundaries[1:]:
        prev_s, prev_e = merged[-1]
        if s - prev_e <= merge_gap:
            merged[-1][1] = max(prev_e, e)
        else:
            merged.append([s, e])

    return merged

Choosing the Right VAD

Tool Best For Pros Cons
Silero VAD Short segments, real-time Better short-segment detection Requires PyTorch
SpeechBrain VAD General purpose Reliable, well-tested May miss short segments
WebRTC VAD Lightweight apps Fast, lightweight Less accurate, requires specific frame sizes

Common Issues and Solutions

  1. Too many false alarms: Increase threshold or min_speech_duration_ms
  2. Missing short segments: Use Silero VAD or decrease threshold
  3. Over-segmentation: Increase merge_gap in postprocessing
  4. Missing speech at boundaries: Decrease min_silence_duration_ms

Integration with Speaker Diarization

VAD boundaries are used to:

  1. Extract speech segments for speaker embedding extraction
  2. Filter out non-speech regions
  3. Improve clustering by focusing on actual speech
python
# After VAD, extract embeddings only for speech segments
for start, end in vad_boundaries:
    segment_audio = waveform[:, int(start*sr):int(end*sr)]
    embedding = speaker_model.encode_batch(segment_audio)
    # ... continue with clustering

Expand your agent's capabilities with these related and highly-rated skills.

benchflow-ai/skillsbench

csv-processing

Use this skill when reading sensor data from CSV files, writing simulation results to CSV, processing time-series data with pandas, or handling missing values in datasets.

897 232
Explore
benchflow-ai/skillsbench

pid-controller

Use this skill when implementing PID control loops for adaptive cruise control, vehicle speed regulation, throttle/brake management, or any feedback control system requiring proportional-integral-derivative control.

897 232
Explore
benchflow-ai/skillsbench

yaml-config

Use this skill when reading or writing YAML configuration files, loading vehicle parameters, or handling config file parsing with proper error handling.

897 232
Explore
benchflow-ai/skillsbench

simulation-metrics

Use this skill when calculating control system performance metrics such as rise time, overshoot percentage, steady-state error, or settling time for evaluating simulation results.

897 232
Explore
benchflow-ai/skillsbench

vehicle-dynamics

Use this skill when simulating vehicle motion, calculating safe following distances, time-to-collision, speed/position updates, or implementing vehicle state machines for cruise control modes.

897 232
Explore
benchflow-ai/skillsbench

web-interface-guidelines

Vercel's comprehensive UI guidelines for building accessible, performant web interfaces. Use this skill when reviewing or building UI components for compliance with best practices around accessibility, performance, animations, and visual stability.

897 232
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results