Agent skill
feature_engineering
Engineer dataset features before ML or Causal Inference. Methods include encoding categorical variables, scaling numerics, creating interactions, and selecting relevant features.
Install this agent skill to your Project
npx add-skill https://github.com/benchflow-ai/skillsbench/tree/main/tasks-no-skills/trend-anomaly-causal-inference/environment/skills/feature_engineering
SKILL.md
Feature Engineering Framework
Comprehensive, modular feature engineering framework general tabular datasets. Provides strategy-based operations including numerical scaling, categorical encoding, polynomial features, and feature selection through a configurable pipeline.
Core Components
FeatureEngineeringStrategies
Collection of static methods for feature engineering operations:
Numerical Features (if intepretability is not a concern)
scale_numerical(df, columns, method)- Scale using 'standard', 'minmax', or 'robust'create_bins(df, columns, n_bins, strategy)- Discretize using 'uniform', 'quantile', or 'kmeans'create_polynomial_features(df, columns, degree)- Generate polynomial and interaction termscreate_interaction_features(df, column_pairs)- Create multiplication interactionscreate_log_features(df, columns)- Log-transform for skewed distributions
Categorical Features
encode_categorical(df, columns, method)- Encode using 'onehot', 'label', 'frequency', or 'hash'create_category_aggregations(df, categorical_col, numerical_cols, agg_funcs)- Group-level statistics
Binary Features
convert_to_binary(df, columns)- Convert Yes/No, True/False to 0/1 (data type to int)
Data Quality Validation
validate_numeric_features(df, exclude_cols)- Verify all features are numeric (except ID columns)validate_no_constants(df, exclude_cols)- Remove constant columns with no variance
Feature Selection
select_features_variance(df, columns, threshold)- Remove low-variance features (default: 0.01). For some columns that consist of almost the same values, we might consider to drop due to the low variance it brings in order to reduce dimensionality.select_features_correlation(df, columns, threshold)- Remove highly correlated features
FeatureEngineeringPipeline
Orchestrates multiple feature engineering steps with logging.
CRITICAL REQUIREMENTS:
- ALL output features MUST be numeric (int or float) - DID analysis cannot use string/object columns
- Preview data types BEFORE processing:
df.dtypesanddf.head()to check actual values - Encode ALL categorical variables - strings like "degree", "age_range" must be converted to numbers
- Verify output: Final dataframe should have
df.select_dtypes(include='number').shape[1] == df.shape[1] - 1(excluding ID column)
Usage Example
from feature_engineering import FeatureEngineeringStrategies, FeatureEngineeringPipeline
# Create pipeline
pipeline = FeatureEngineeringPipeline(name="Demographics")
# Add feature engineering steps
pipeline.add_step(
FeatureEngineeringStrategies.convert_to_binary,
columns=['<column5>', '<column2>'],
description="Convert binary survey responses to 0/1"
).add_step(
FeatureEngineeringStrategies.encode_categorical,
columns=['<column3>', '<column7>'],
method='onehot',
description="One-hot encode categorical features"
).add_step(
FeatureEngineeringStrategies.scale_numerical,
columns=['<column10>', '<column1>'],
method='standard',
description="Standardize numerical features"
).add_step(
FeatureEngineeringStrategies.validate_numeric_features,
exclude_cols=['<ID Column>'],
description="Verify all features are numeric before modeling"
).add_step(
FeatureEngineeringStrategies.validate_no_constants,
exclude_cols=['<ID Column>'],
description="Remove constant columns with no predictive value"
).add_step(
FeatureEngineeringStrategies.select_features_variance,
columns=[], # Empty = auto-select all numerical
threshold=0.01,
description="Remove low-variance features"
)
# Execute pipeline
# df_complete: complete returns original columns and the engineered features
df_complete = pipeline.execute(your_cleaned_df, verbose=True)
# Shortcut: Get the ID Column with the all needed enigneered features
engineered_features = pipeline.get_engineered_features()
df_id_pure_features = df_complete[['<ID Column>']+engineered_features]
# Get execution log
log_df = pipeline.get_log()
Input
- A valid dataFrame that would be sent to feature engineering after any data processing, imputation, or drop (A MUST)
Output
- DataFrame with both original and engineered columns
- Engineered feature names accessible via
pipeline.get_engineered_features() - Execution log available via
pipeline.get_log()
Key Features
- Multiple encoding methods for categorical variables
- Automatic handling of high-cardinality categoricals
- Polynomial and interaction feature generation
- Built-in feature selection for dimensionality reduction
- Pipeline pattern for reproducible transformations
Best Practices
- Always validate data types before downstream analysis: Use
validate_numeric_features()after encoding - Check for constant columns that provide no information: Use
validate_no_constants()before modeling - Convert binary features before other transformations
- Use one-hot encoding for low-cardinality categoricals
- Use KNN imputation if missing value could be inferred from other relevant columns
- Use hash encoding for high-cardinality features (IDs, etc.)
- Apply variance threshold to remove constant features
- Check correlation matrix before modeling to avoid multicollinearity
- MAKE SURE ALL ENGINEERED FEATURES ARE NUMERICAL
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
csv-processing
Use this skill when reading sensor data from CSV files, writing simulation results to CSV, processing time-series data with pandas, or handling missing values in datasets.
pid-controller
Use this skill when implementing PID control loops for adaptive cruise control, vehicle speed regulation, throttle/brake management, or any feedback control system requiring proportional-integral-derivative control.
yaml-config
Use this skill when reading or writing YAML configuration files, loading vehicle parameters, or handling config file parsing with proper error handling.
simulation-metrics
Use this skill when calculating control system performance metrics such as rise time, overshoot percentage, steady-state error, or settling time for evaluating simulation results.
vehicle-dynamics
Use this skill when simulating vehicle motion, calculating safe following distances, time-to-collision, speed/position updates, or implementing vehicle state machines for cruise control modes.
web-interface-guidelines
Vercel's comprehensive UI guidelines for building accessible, performant web interfaces. Use this skill when reviewing or building UI components for compliance with best practices around accessibility, performance, animations, and visual stability.
Didn't find tool you were looking for?