DeepBuilder

DeepBuilder is an end-to-end autoresearch agent framework for productive, stable, and goal-aligned research over hours or days. The current implementation centers on a recoverable executable workspace: agents inspect task data, write and run code, evaluate candidate artifacts, preserve validated stages, reconsider research direction, and resume interrupted work without discarding established evidence.

Overview

DeepBuilder Architecture — **DeepBuilder system architecture.** Research workers operate over recoverable executable states and adapt long-horizon trajectories through boundary-triggered ESTRA transitions, while evidence-aware execution control coordinates physical resource allocation and runtime execution.

On OpenAI MLE-bench, DeepBuilder ranked #1, demonstrating strong performance on real-world AI model building tasks.

Across machine learning, scientific modeling, and mathematical optimization, DeepBuilder sustains effective long-horizon research and reaches 70.22 ± 1.18% Any-Medal on the full 75-task MLE-bench within a 24-hour budget, exceeding the strongest reported baseline by 5.78 percentage points.

The same runtime supports each of these domains. Task-specific artifact schemas and metrics live in registered task packages, while the core runtime remains task-domain agnostic.

ℹ️ Note
The authoritative candidate path is: GateService → evaluator backend → MetricEvent → GatePolicy → GateDecision
An evaluator result is evidence; only an accepted Gate decision can create a Stage.

💡 Quickstart
Install DeepBuilder via pip:
pip install deepbuilder

End-to-End Flow

CLI / Manifesttask, budget, workspace, resources

Workspace Setupflat dataset and optional prior state

LNR Workerspersistent ScienceAgent sessions

Gate & Evaluatorartifact evidence and admission

Stage & Snapshotdurable result and source state

ESTRA & Foldingcontinue, redirect, or restore

Global Mergevalidated final artifacts

Resource control runs alongside this flow. Bash jobs publish execution facts to a task-local runtime that manages CPU boundaries, GPU leases, queues, monitored trials, review state, bounded timeboxes, and guarded intervention. The control plane manages execution opportunity and efficiency; the research agent remains responsible for scientific route selection.

Module Map

CLI & Configuration: Click commands, modular defaults, manifests, profiles, evaluator settings, and resource-control presets.
Data Preparation: Agent-driven preparation of a flat dataset/ view and auditable train/validation split artifacts.
Core Runtime: Orchestration, task packages, parallel task isolation, model routing, tracing, and process cleanup.
Agent & Tools: The ScienceAgent run loop, tool execution, workspace guards, output compression, and memory projection.
LNR Solver: Long-horizon workers, Stage transactions, snapshots, ESTRA, stage-memory folding, resume, and final merge.
Resource Management: Task-local admission, lease and queue control, review state, advisory, arbiter, strict gate, and feedback hygiene.
Skills & Safety: Markdown skill discovery plus execution, code, leakage, stdout, and resource safety evidence.
Selection & Observability: Stage/evaluator/resource event stores, score summaries, monitors, traces, and finals.
Task Onboarding: The current task-package contract for optimization and scientific-modeling tasks.

Current Repository Boundaries

DeepBuilder/
├── deepbuilder/
│   ├── core/                 # Agent, tools, orchestration, task registry
│   ├── gates/                # Gate policies and evaluator plugins
│   ├── safety/               # Execution, code, leakage, stdout, resource safety
│   ├── solver/lnr/           # Long-horizon runtime, stages, ESTRA, resume, merge
│   └── ui/                   # Live monitor and HTML trace renderer
├── tasks/
│   ├── ml/mlebench/          # MLE-bench task packages
│   ├── opt_solver/           # Optimization task packages
│   └── sci_modeling_bench/   # Scientific-modeling task packages
├── scripts/                  # Maintained example manifests and monitor helpers
├── .deepbuilder/skills/      # Repository skill library
└── doc/deepbuilder/          # This documentation set

Stable Contracts

Contract	Current implementation
Task identity	`tasks/**/task.yaml` declares task id, category, provider/profile, description, artifact, metric, evaluator, and optional Gate policy.
Candidate admission	`deepbuilder/gates/service.py` owns evaluation and policy application. Primary mode fails closed on missing, duplicate, or invalid outcomes.
Validated progress	An accepted candidate is recorded in the hidden ledger, materialized as a Stage, captured in a workspace snapshot, and summarized in `task_logs/lhr_stage_performance.csv`.
Long-term memory	The append-only ledger remains the source of truth. `stage_memory.py` folds older cards into persistent summaries while retaining recent and verification-critical raw cards.
Recovery	Task resume reuses the workspace and budget state; agent resume reloads persisted memory; a single pending tail tool call can be replayed directly.
Finalization	`solver/lnr/global_merge/` evaluates worker evidence and writes one or more final directories under `merge/finals/final_*`.

Runtime Artifact Layout

task_root/
├── task_logs/
│   ├── lhr_stage_performance.csv
│   ├── lhr_events.jsonl
│   ├── state.json
│   └── resource/resource_events.jsonl
├── workers/
│   └── w00/
│       ├── workspace/
│       ├── snapshots/
│       └── logs/
│           ├── evaluator_events.jsonl
│           └── deepbuilder_time_trace.csv
└── merge/
    ├── global_merge_manifest.json
    └── finals/final_*/