Agent skill

machine-learning

Machine learning development patterns, model training, evaluation, and deployment. Use when building ML pipelines, training models, feature engineering, model evaluation, or deploying ML systems to production.

View SKILL.md on GitHub Repository
Stars 4
Forks 0

Install this agent skill to your Project

npx add-skill https://github.com/89jobrien/steve/tree/main/steve/skills/machine-learning

SKILL.md

Machine Learning

Comprehensive machine learning skill covering the full ML lifecycle from experimentation to production deployment.

When to Use This Skill

  • Building machine learning pipelines
  • Feature engineering and data preprocessing
  • Model training, evaluation, and selection
  • Hyperparameter tuning and optimization
  • Model deployment and serving
  • ML experiment tracking and versioning
  • Production ML monitoring and maintenance

ML Development Lifecycle

1. Problem Definition

Classification Types:

  • Binary classification (spam/not spam)
  • Multi-class classification (image categories)
  • Multi-label classification (document tags)
  • Regression (price prediction)
  • Clustering (customer segmentation)
  • Ranking (search results)
  • Anomaly detection (fraud detection)

Success Metrics by Problem Type:

Problem Type Primary Metrics Secondary Metrics
Binary Classification AUC-ROC, F1 Precision, Recall, PR-AUC
Multi-class Macro F1, Accuracy Per-class metrics
Regression RMSE, MAE R², MAPE
Ranking NDCG, MAP MRR
Clustering Silhouette, Calinski-Harabasz Davies-Bouldin

2. Data Preparation

Data Quality Checks:

  • Missing value analysis and imputation strategies
  • Outlier detection and handling
  • Data type validation
  • Distribution analysis
  • Target leakage detection

Feature Engineering Patterns:

  • Numerical: scaling, binning, log transforms, polynomial features
  • Categorical: one-hot, target encoding, frequency encoding, embeddings
  • Temporal: lag features, rolling statistics, cyclical encoding
  • Text: TF-IDF, word embeddings, transformer embeddings
  • Geospatial: distance features, clustering, grid encoding

Train/Test Split Strategies:

  • Random split (standard)
  • Stratified split (imbalanced classes)
  • Time-based split (temporal data)
  • Group split (prevent data leakage)
  • K-fold cross-validation

3. Model Selection

Algorithm Selection Guide:

Data Size Problem Recommended Models
Small (<10K) Classification Logistic Regression, SVM, Random Forest
Small (<10K) Regression Linear Regression, Ridge, SVR
Medium (10K-1M) Classification XGBoost, LightGBM, Neural Networks
Medium (10K-1M) Regression XGBoost, LightGBM, Neural Networks
Large (>1M) Any Deep Learning, Distributed training
Tabular Any Gradient Boosting (XGBoost, LightGBM, CatBoost)
Images Classification CNN, ResNet, EfficientNet, Vision Transformers
Text NLP Transformers (BERT, RoBERTa, GPT)
Sequential Time Series LSTM, Transformer, Prophet

4. Model Training

Hyperparameter Tuning:

  • Grid Search: exhaustive, good for small spaces
  • Random Search: efficient, good for large spaces
  • Bayesian Optimization: smart exploration (Optuna, Hyperopt)
  • Early stopping: prevent overfitting

Common Hyperparameters:

Model Key Parameters
XGBoost learning_rate, max_depth, n_estimators, subsample
LightGBM num_leaves, learning_rate, n_estimators, feature_fraction
Random Forest n_estimators, max_depth, min_samples_split
Neural Networks learning_rate, batch_size, layers, dropout

5. Model Evaluation

Evaluation Best Practices:

  • Always use held-out test set for final evaluation
  • Use cross-validation during development
  • Check for overfitting (train vs validation gap)
  • Evaluate on multiple metrics
  • Analyze errors qualitatively

Handling Imbalanced Data:

  • Resampling: SMOTE, undersampling
  • Class weights: weighted loss functions
  • Threshold tuning: optimize decision threshold
  • Evaluation: use PR-AUC over ROC-AUC

6. Production Deployment

Model Serving Patterns:

  • REST API (Flask, FastAPI, TF Serving)
  • Batch inference (scheduled jobs)
  • Streaming (real-time predictions)
  • Edge deployment (mobile, IoT)

Production Considerations:

  • Latency requirements (p50, p95, p99)
  • Throughput (requests per second)
  • Model size and memory footprint
  • Fallback strategies
  • A/B testing framework

7. Monitoring & Maintenance

What to Monitor:

  • Prediction latency
  • Input feature distributions (data drift)
  • Prediction distributions (concept drift)
  • Model performance metrics
  • Error rates and types

Retraining Triggers:

  • Performance degradation below threshold
  • Significant data drift detected
  • Scheduled retraining (daily, weekly)
  • New training data available

MLOps Best Practices

Experiment Tracking

Track for every experiment:

  • Code version (git commit)
  • Data version (hash or version ID)
  • Hyperparameters
  • Metrics (train, validation, test)
  • Model artifacts
  • Environment (packages, versions)

Model Versioning

models/
├── model_v1.0.0/
│   ├── model.pkl
│   ├── metadata.json
│   ├── requirements.txt
│   └── metrics.json
├── model_v1.1.0/
└── model_v2.0.0/

CI/CD for ML

  1. Continuous Integration:

    • Data validation tests
    • Model training tests
    • Performance regression tests

  2. Continuous Deployment:

    • Staging environment validation
    • Shadow mode testing
    • Gradual rollout (canary)
    • Automatic rollback

Reference Files

For detailed patterns and code examples, load reference files as needed:

  • references/preprocessing.md - Data preprocessing patterns and feature engineering techniques
  • references/model_patterns.md - Model architecture patterns and implementation examples
  • references/evaluation.md - Comprehensive evaluation strategies and metrics

Integration with Other Skills

  • performance - For optimizing inference latency
  • testing - For ML-specific testing patterns
  • database-optimization - For feature store queries
  • debugging - For model debugging and error analysis

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

89jobrien/steve

python-scripting

Python scripting with uv and PEP 723 inline dependencies. Use when creating standalone Python scripts with automatic dependency management.

4 0
Explore
89jobrien/steve

example-skill

This skill should be used when demonstrating skill structure and format. Provides example patterns for creating new skills.

4 0
Explore
89jobrien/steve

tool-presets

Standardized tool set definitions for Claude Code agents ensuring consistent tool access across similar agent types

4 0
Explore
89jobrien/steve

file-organizer

Intelligently organizes your files and folders across your computer by understanding context, finding duplicates, suggesting better structures, and automating cleanup tasks. Reduces cognitive load and keeps your digital workspace tidy without manual effort.

4 0
Explore
89jobrien/steve

brainstorming

You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation.

4 0
Explore
89jobrien/steve

database-optimization

SQL query optimization and database performance specialist. Use when optimizing slow queries, fixing N+1 problems, designing indexes, implementing caching, or improving database performance. Works with PostgreSQL, MySQL, and other databases.

4 0
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results