Agent skill
exps-logistic
Documentation for the logistic regression MI estimation experiment (exps_logistic)
Install this agent skill to your Project
npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/other/exps-logistic
SKILL.md
exps_logistic - Logistic Regression MI Estimation
Purpose
Measure mutual information (MI) lower bound between CoT rationales and problem parameters using multinomial logistic regression on embeddings. Compares code vs NL representations.
Key Files
| File | Purpose |
|---|---|
src/exps_logistic/main.py |
Main experiment runner |
src/exps_logistic/config.py |
Configuration, CLI args, kind presets |
src/exps_logistic/data_utils.py |
Data loading, gamma label creation |
src/exps_logistic/classifier.py |
Logistic regression classifier |
src/exps_logistic/featurizer.py |
Embedding extraction (BERT, TF-IDF) |
src/exps_logistic/run.sh |
Dev script for single model |
src/exps_logistic/prod_logistic.sh |
Production script for all models |
src/exps_logistic/notebooks/generate_plots.py |
Plot generation |
Kind Presets (in config.py)
FG_KINDS = {"add", "sub", "mul", "lcs", "knap", "rod", "ilp_assign", "ilp_prod", "ilp_partition"}
CLRS_KINDS = {
"activity_selector", "articulation_points", "bellman_ford", "bfs",
"binary_search", "bridges", "bubble_sort", "dag_shortest_paths",
"dfs", "dijkstra", "find_maximum_subarray_kadane", "floyd_warshall",
"graham_scan", "heapsort", "insertion_sort", "jarvis_march",
"kmp_matcher", "lcs_length", "matrix_chain_order", "minimum",
"mst_kruskal", "mst_prim", "naive_string_matcher", "optimal_bst",
"quickselect", "quicksort", "segments_intersect",
"strongly_connected_components", "task_scheduling", "topological_sort",
}
NPHARD_KINDS = {"edp", "gcp", "ksp", "spp", "tsp"}
EXTENDED_KINDS = FG_KINDS | CLRS_KINDS | NPHARD_KINDS # 44 total kinds
Gamma Labels
Format: {kind}|d{digits}|b{bin} (e.g., knap|d8|b24)
- Fine-grained kinds have parsed bin values based on problem parameters
- CLRS/NP-hard kinds get
bNA(no fine-grained parsing)
CLI Arguments
| Arg | Description |
|---|---|
--results-dir |
Path to exps_performance results |
--models |
Model filter (e.g., anthropic/claude-haiku-4.5) |
--rep |
Representation: code or nl |
--label |
Label type: kind, theta_new, or gamma |
--kinds-preset |
Kind filter: fg, clrs, nphard, or extended |
--feats |
Featurizer: hf-cls, tfidf, st, openai |
--embed-model |
HuggingFace model for embeddings |
--no-cv |
Disable cross-validation |
--bits |
Report cross-entropy in bits |
Generated Plots (in src/exps_logistic/notebooks/)
| Plot | Description |
|---|---|
boxplot_extended.png |
Code vs NL MI comparison |
kde_extended.png |
KDE density comparison |
mi_vs_accuracy_extended.png |
MI vs task accuracy correlation |
contrast_extended.png |
Per-model contrast (code - nl) |
label_dist_by_kind.png |
Sample counts by problem kind |
label_dist_by_digits.png |
Fine-grained distribution by digits |
label_dist_heatmap.png |
Kind × Digits heatmap |
label_dist_top30.png |
Top 30 most frequent labels |
label_dist_zipf.png |
Zipf-like frequency distribution |
label_dist_pie.png |
Category breakdown pie chart |
Key Results (Extended Kinds - Jan 9, 2026)
- 28 successful runs across 6 models
- Pearson r = 0.486, p = 0.009 (MI vs accuracy correlation)
- 500 unique labels across 44 problem kinds
- Code representation shows higher MI lower bounds than NL
- Models tested: claude-haiku-4.5, gemini-2.5-flash, gpt-4o-mini, llama-3.1-405b-instruct, ministral-14b-2512, qwen-2.5-coder-32b-instruct
Running Commands
# Dev run (single model)
bash src/exps_logistic/run.sh
# Production run (all models, all seeds)
bash src/exps_logistic/prod_logistic.sh
# Generate plots
uv run --no-sync python src/exps_logistic/notebooks/generate_plots.py
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
Didn't find tool you were looking for?