Agent skill
parallel-processing
Parallel processing with joblib for grid search and batch computations. Use when speeding up computationally intensive tasks across multiple CPU cores.
Install this agent skill to your Project
npx add-skill https://github.com/benchflow-ai/skillsbench/tree/main/tasks/mars-clouds-clustering/environment/skills/parallel-processing
SKILL.md
Parallel Processing with joblib
Speed up computationally intensive tasks by distributing work across multiple CPU cores.
Basic Usage
from joblib import Parallel, delayed
def process_item(x):
"""Process a single item."""
return x ** 2
# Sequential
results = [process_item(x) for x in range(100)]
# Parallel (uses all available cores)
results = Parallel(n_jobs=-1)(
delayed(process_item)(x) for x in range(100)
)
Key Parameters
- n_jobs:
-1for all cores,1for sequential, or specific number - verbose:
0(silent),10(progress),50(detailed) - backend:
'loky'(CPU-bound, default) or'threading'(I/O-bound)
Grid Search Example
from joblib import Parallel, delayed
from itertools import product
def evaluate_params(param_a, param_b):
"""Evaluate one parameter combination."""
score = expensive_computation(param_a, param_b)
return {'param_a': param_a, 'param_b': param_b, 'score': score}
# Define parameter grid
params = list(product([0.1, 0.5, 1.0], [10, 20, 30]))
# Parallel grid search
results = Parallel(n_jobs=-1, verbose=10)(
delayed(evaluate_params)(a, b) for a, b in params
)
# Filter results
results = [r for r in results if r is not None]
best = max(results, key=lambda x: x['score'])
Pre-computing Shared Data
When all tasks need the same data, pre-compute it once:
# Pre-compute once
shared_data = load_data()
def process_with_shared(params, data):
return compute(params, data)
# Pass shared data to each task
results = Parallel(n_jobs=-1)(
delayed(process_with_shared)(p, shared_data)
for p in param_list
)
Performance Tips
- Only worth it for tasks taking >0.1s per item (overhead cost)
- Watch memory usage - each worker gets a copy of data
- Use
verbose=10to monitor progress
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
csv-processing
Use this skill when reading sensor data from CSV files, writing simulation results to CSV, processing time-series data with pandas, or handling missing values in datasets.
pid-controller
Use this skill when implementing PID control loops for adaptive cruise control, vehicle speed regulation, throttle/brake management, or any feedback control system requiring proportional-integral-derivative control.
yaml-config
Use this skill when reading or writing YAML configuration files, loading vehicle parameters, or handling config file parsing with proper error handling.
simulation-metrics
Use this skill when calculating control system performance metrics such as rise time, overshoot percentage, steady-state error, or settling time for evaluating simulation results.
vehicle-dynamics
Use this skill when simulating vehicle motion, calculating safe following distances, time-to-collision, speed/position updates, or implementing vehicle state machines for cruise control modes.
web-interface-guidelines
Vercel's comprehensive UI guidelines for building accessible, performant web interfaces. Use this skill when reviewing or building UI components for compliance with best practices around accessibility, performance, animations, and visual stability.
Didn't find tool you were looking for?