SeekBrain is a domain-specialized multi-agent AI system that turns neuroscience datasets and analysis questions into executable code pipelines, scored against expert rubrics on the BrainArena benchmark. On 32 expert-annotated tasks spanning zebrafish, mouse, macaque, fly, and worm data, SeekBrain averaged 75.8 points out of 100, compared with 64.1 for Claude Code and 58.1 for OpenAI Codex (arXiv:2607.29347). The framework extracts analysis recipes from code-paper pairs, plans hierarchical workflows, and executes cross-modal analyses that general coding agents often mishandle. For teams following AI research on scientific agents, or exploring AI research tooling, SeekBrain illustrates what changes when domain rubrics replace generic "write Python" prompts.
Why Neuroscience Needs Specialized Agents
Neuroscience analysis is fragmented across species, modalities, file formats, and statistical conventions that generic LLM coding agents rarely internalize without domain grounding. A mouse two-photon calcium imaging dataset, a larval zebrafish whole-brain light-sheet recording, and an International Brain Laboratory (IBL) behavioral electrophysiology release each demand different preprocessing, alignment, and hypothesis tests. Misspecifying bout definitions, brain atlas coordinates, or decoder cross-validation splits produces plausible-looking figures with invalid conclusions.
SeekBrain addresses the mismatch by maintaining a co-evolving Neuroscience Analysis Repertoire: workflows distilled from published code associated with peer-reviewed papers. Agent planners select and compose recipes rather than inventing analysis steps from scratch. When researchers steer the agent with high-level questions ("cluster swimming bouts and link clusters to brain-wide activity"), SeekBrain expands instructions into script generation, execution, plotting, and interpretation grounded in repertoire patterns.
Ablation experiments removing the repertoire dropped mean BrainArena scores from 75.8 to 63.2, demonstrating that codified domain expertise, not raw LLM scale alone, drives performance gains.
SeekBrain's architecture separates a Research Planning Engine (hierarchical task decomposition), an Analysis Repertoire (codified workflows from code-paper pairs), and execution engines that run Python in sandboxed environments. The repertoire co-evolves as new analyses succeed: validated recipes enter the library for future tasks. This design differs from generic "AI Scientist" frameworks that propose hypotheses without executable pipelines on native neuroscience file formats (HDF5 calcium movies, NWB electrophysiology, atlas-registered coordinates).
Comparison baselines on BrainArena received only the analysis query and raw data paths, blinded to reference figures and published conclusions. SeekBrain used Claude Opus 4.7 as the base LLM for most agents, matching the Claude Code baseline model class. Fair comparison isolates domain scaffolding rather than model scale differences alone.
BrainArena Rubric and Scoring Dimensions
BrainArena evaluates agents on 32 tasks derived from ten published neuroscience studies, each scored with a 100-point expert rubric across figure fidelity, method correctness, and interpretive validity. Figure fidelity checks whether visual patterns match reference figures from source papers: cluster geometry, axis ranges, color conventions, and effect sizes. Method correctness verifies that statistical tests, preprocessing order, and hyperparameters align with reproducible neuroscience practice. Interpretive validity judges whether the agent's written conclusions follow logically from results without overclaiming causation or ignoring confounds.
| Rubric dimension | What it measures | SeekBrain advantage |
|---|---|---|
| Figure fidelity | Visual match to published reference figures | Strong; repertoire encodes plotting conventions |
| Method correctness | Valid pipelines, splits, and statistics | Largest gap on hard tasks vs Codex |
| Interpretive validity | Scientifically sound conclusions | Biggest relative edge over Claude Code |
Rubrics contain over 700 deduction subitems mapping observable errors to point losses. Tasks span seven analysis types: neural response characterization, decoding, population representation analysis, computational modeling, statistical analysis, brain network mapping, and molecular or developmental analysis. Organism coverage includes zebrafish (31% of tasks), mouse (34%), macaque (12%), C. elegans (16%), and Drosophila (6%). Most tasks integrate multiple modalities; behavioral plus neural activity is the most common pairing at 53%.
Difficulty splits into 12 easy and 20 hard tasks. SeekBrain's margin widened on hard tasks, where method correctness and interpretive validity penalties accumulate. Benchmark designers blinded agents to source paper identities, reference figures, and ground-truth conclusions, supplying only task queries and raw data paths.
Statistical testing used Friedman tests across tasks with Dunn post hoc comparisons. SeekBrain versus Claude Code: adjusted p = 0.0002. SeekBrain versus Codex: adjusted p less than 0.0001. Component-level rubric breakdowns showed SeekBrain's largest advantage on interpretive validity, the dimension assessing whether written conclusions respect association versus causation, acknowledge limitations, and use correct neuroanatomical terminology.
BrainArena complements general benchmarks like Humanity's Last Exam (broad academic reasoning) and SWE-bench (software engineering) by scoring end-to-end scientific workflows on real multimodal data. A coding agent that passes unit tests can still produce biologically meaningless cluster counts or misaligned brain atlas plots; BrainArena penalizes those failures explicitly.
Zebrafish Swimming Bout Case Study
In researcher-steering mode, SeekBrain analyzed free-moving larval zebrafish multimodal data, clustering 1,546 swimming bouts from 18 larvae into three kinematic types and linking them to brain-wide calcium imaging via CEBRA embeddings. The workflow adapted established bout-decomposition methods: principal components of tail posture, polarity-invariant dynamic time warping distances, Isomap embedding, and Ward hierarchical clustering. Silhouette scores peaked at K=3, separating large-angle turns, J-turns, and low-amplitude forward swimming.
Raw datasets contained roughly 64,000 region-of-interest neural traces per fish. SeekBrain grouped ROIs into 3,000 region-aware supervoxels per larva by combining activity principal components with spatial coordinates within anatomical regions. Supervised CEBRA training on behaviorally relevant frames produced eight-dimensional neural representations that separated bout clusters within each fish. A 3-nearest-neighbor classifier achieved 94.9% plus or minus 3.4% cross-validation accuracy decoding bout type from CEBRA embeddings.
Cross-animal alignment via orthogonal Procrustes analysis showed matching cluster centroids sat significantly closer than nonmatching centroids (paired Wilcoxon test, p = 2.44 x 10^-4, n = 13 larvae). Region-aware permutation tests identified tuned supervoxels distributed across 39 to 47 of 81 anatomical regions per cluster, including thalamus, pretectum, hypothalamic subregions, and hindbrain raphe nuclei. The agent generated and executed the full script chain from clustering through brain mapping with researcher prompts at each stage.
IBL Mouse Decoder Reanalysis
In a more autonomous rediscovery task, SeekBrain reanalyzed International Brain Laboratory mouse decision-making decoder maps and identified a shared low-rank axis of regional decoding strength across stimulus, choice, and feedback variables. Prior work established that movement covaries broadly with neural activity across cortex. SeekBrain loaded five published region-wise decoder maps, regressed out speed and velocity components, and tested whether residual spatial patterns shared structure across cognitive variables.
The agent formulated and validated a hypothesis about a common regional decoding axis while preserving spatial variation unique to stimulus, choice, and feedback maps after movement regression. This mode approximates how a postdoctoral researcher might explore a public dataset without step-by-step micromanagement, though human oversight remains essential before publication.
IBL datasets provide standardized behavioral task structure across laboratories, making them attractive rediscovery targets. SeekBrain's ability to regress movement covariates before interpreting cognitive decoder topography addresses a recurring pitfall in systems neuroscience: apparent cognitive maps that partially reflect running speed or pupil-linked arousal. Agents that skip that regression step score poorly on interpretive validity even when figures look polished.
Agent vs Human Workflow
SeekBrain operates in researcher-steering mode (human prompts at each analysis branch) and SeekBrain-led mode (agent proposes hypotheses then executes), mirroring how labs mix directed and exploratory projects. Zebrafish analyses used steering: the human specified bout clustering, supervoxel construction, CEBRA training, and cross-fish alignment steps. IBL reanalysis leaned autonomous: SeekBrain proposed the shared decoding-axis hypothesis and ran supporting statistics. Neither mode eliminates the principal investigator; it shifts time from writing matplotlib boilerplate to evaluating whether agent-generated claims survive biological scrutiny.
Open-source distribution via npm package seekbrain and documented example prompts (structural connectivity modules, bout clustering pipelines) lower the barrier for labs to trial agent-assisted workflows on their own zebrafish or mouse releases without reimplementing BrainArena scoring locally.
Limits of General-Purpose Coding Agents
Claude Code and Codex excel at generic software engineering but lack native neuroscience data schemas, atlas registrations, and interpretive norms unless heavily prompted. BrainArena used Claude Opus 4.7 for SeekBrain's base LLM and matched baselines where applicable. Codex (GPT-5.5 in the paper's configuration) scored lowest overall, with particular weakness on interpretive validity: generating correct-looking plots paired with overstated causal claims or incorrect biological terminology.
Frameworks focused on theoretical hypothesis generation without executable pipelines (for example, some AI Scientist variants) were excluded from direct comparison because they do not ingest native neuroscience file formats. SeekBrain targets reproducible analysis execution, not literature review alone. Open-source release via npm package seekbrain and a project page (ai4neurolab.github.io/SeekBrain-page) supports researcher-in-the-loop workflows rather than fully unsupervised discovery.
Limitations remain: BrainArena tasks derive from published studies with public data; novel unpublished modalities may lack repertoire coverage. Agents can still hallucinate package APIs or mis-specify random seeds. Expert rubrics encode consensus interpretations that future papers might dispute. SeekBrain accelerates routine multimodal analysis; it does not replace experimental design, animal welfare oversight, or peer review.
Reproducibility expectations in neuroscience (preregistered analyses, shared code, version-pinned dependencies) apply equally to agent outputs. Labs should archive SeekBrain-generated scripts alongside human-written notebooks in OSF or Zenodo deposits. BrainArena's inclusion of a bioRxiv preprint task published after Claude's training cutoff helps detect benchmark contamination from memorized paper text, though agents could still exploit leaked figure descriptions if prompts were careless.
Frequently Asked Questions
What is the BrainArena benchmark?
BrainArena is an expert-annotated evaluation suite of 32 neuroscience data analysis tasks drawn from ten source publications, covering five model organisms and multiple modalities. Each task includes a query, data paths, reference figures, ground-truth conclusions, and a 100-point rubric with over 700 deduction subitems.
How does SeekBrain score versus Codex and Claude Code?
Mean scores: SeekBrain 75.8, Claude Code 64.1, Codex 58.1. Friedman test p less than 0.0001; post hoc Dunn tests showed SeekBrain significantly outperformed both baselines. Without the analysis repertoire, SeekBrain dropped to 63.2.
Can SeekBrain access my private lab data?
SeekBrain runs locally against paths you provide; it does not inherently upload data to external servers unless configured to call cloud LLM APIs. Institutional policies on sending neural recordings to third-party models still apply. Check IRB and data-use agreements before running patient or proprietary datasets through any agent.
Are SeekBrain analyses reproducible?
The framework logs generated scripts and execution steps, supporting audit trails. Reproducibility still depends on fixed random seeds, package versions, and identical raw data hashes. Researchers should version-control agent outputs like any computational notebook.
What species does SeekBrain support?
BrainArena tasks cover zebrafish, mouse, macaque, Drosophila, and C. elegans. The repertoire can expand as new code-paper pairs are added; coverage outside benchmarked organisms may be thin until recipes exist.
Where is SeekBrain published?
"SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery" (arXiv:2607.29347). Code and documentation are linked from the project page maintained by the AI4NeuroLab group.
Should labs replace postdocs with SeekBrain?
No. SeekBrain automates repetitive analysis scaffolding and figure reproduction, freeing researchers for experimental design, causal inference, and theory. Interpretive validity rubrics explicitly penalize agents that confuse correlation with mechanism. Human domain expertise remains the quality gate for AI research outputs destined for publication.
