Overmind fine-tunes small models from code-agent traces, no ML infrastructure needed

A new platform called Overmind takes a different approach to model fine-tuning. Instead of training a large model from scratch, it ingests production traces from existing code agents — whether they run in Cursor, Claude Code, OpenCode, or Codex — and distills them into a small, task-specific model. The SDK and CLI ship under an MIT license (`pip install overmind`); the platform behind them is AGPL-3.0 with a full self-hosting option. A single unified API serves the result, and the weights stay with the developer for download, retraining, or rollback.

In practice the pipeline builds a context graph from code and traces, curates training and evaluation datasets, then fine-tunes a smaller model measured against the developer's own evaluation metrics. The process runs continuously: Overmind "keeps training and improving your agents continuously, with data from your production traces," as the developers put it. The developer picks a code agent on the entry screen, pastes the onboarding prompt into the chat with the repo open, and the tool runs `overmind init` and `overmind sync` to build the graph. From there every action is a `/overmind` command in the same chat.

The hosted version runs at console.overmindlab.ai. Teams that need to keep traces and training data inside their own network can spin up the same stack: `git clone`, copy `.env.example` to `.env` with the required keys, then `docker compose up -d`. That brings up Postgres, Redis, the API on port 8000, the console on port 5173, Celery workers, and Grafana on port 3001. On first boot the API runs migrations and seeds built-in evaluators; Swagger documentation lives at `/api/docs/`. The command `docker compose exec api python manage.py seed_demo --owner <email>` loads a complete demo environment.

The platform itself runs in containers, but fine-tuning and model serving require separate Modal workers: `modal deploy` across three files — `modal_vllm_worker.py`, `register_model.py`, and `modal_sft_worker.py` — plus a Modal secret named `overmind-inference` that holds AWS credentials, `INFERENCE_API_KEY`, and `HF_TOKEN`. Without them the API refuses to start and reports exactly which key is missing. The architecture separates the control plane from the heavy compute plane, so the API can run on a modest machine while GPUs work in the cloud.

The `overmind[tracing]` library adds automatic instrumentation for LLM SDKs via OpenTelemetry when `providers="auto"`. With no API key tracing stays off and the code does not break. Any OTel exporter can POST to `/api/v1/traces`, and connectors sync existing traces from Langfuse, LangSmith, Braintrust, or Galileo. In the UI, Observability → Task executions shows the trace and its score.

The platform exposes an MCP server at `/api/mcp/` with tools, resources, and prompts for agent workflows. Account-level API keys and OAuth connections gain access to every active project the account is authorized for; they call `list_projects` and pass `project_id` in tools and resources, with membership verified on every action. Project-scoped API keys remain limited to their own project. Every tool declares its run cost — free, compute, LLM, or GPU — and no tool can delete anything. An optional add-on package bundles the MCP connection, Overmind branding, and nine workflow skills; access tokens expire after one hour and refresh automatically.