Weco AI

Weco AI: an agent that experiments on your code until the number goes up

Weco AI builds what it calls an AI research engineer: an agent that takes a piece of code, an evaluation command, and a metric — accuracy, latency, throughput, cost — and then autonomously runs experiment after experiment to improve that metric. Its core technology, AIDE (AI-Driven Exploration in the space of code), treats optimization as a tree search: every candidate solution becomes a node, the LLM proposes targeted improvements to promising nodes, and automated evaluations decide which branches to grow. That structure lets it accumulate small verified wins instead of wandering like a chat-based agent.

In practice you install the open-source weco CLI, point it at your training script, GPU kernel, feature pipeline, or prompt, and tell it what to maximize. The agent edits code, runs your eval, reads the result, and iterates — for hours or days if needed — with a human able to inspect the experiment tree at any point. The approach was validated publicly when AIDE became the reference scaffold in frontier-lab agent evaluations for machine learning engineering.

Key Features

• AIDE agent: tree search over code with LLM-proposed improvements and hard-coded search over verified evaluations
• Optimizes against any user-defined metric: model accuracy, GPU kernel speed, inference cost, prompt performance
• Open-source weco CLI plus the original open-source aideml research agent
• Works on real workloads: PyTorch/Triton/CUDA kernels, ML model development, feature engineering, prompt engineering
• Full experiment-tree visibility so engineers can audit what was tried and why
• Published research: AIDE paper (arXiv 2502.13138) and benchmark results on MLE-Bench and RE-Bench

Ideal Use Case

Weco fits ML engineers, researchers, and performance engineers with a measurable objective and a working evaluation harness: squeezing latency out of a GPU kernel, pushing a model past a leaderboard baseline, or tuning a feature or prompt pipeline where each manual experiment costs an afternoon. It is not a general coding copilot — if you cannot express your goal as a metric the agent can run and read, you will not get much out of it.

How Weco AI differentiates

While most coding agents optimize for plausible diffs, AIDE optimizes for measured outcomes, and the receipts are public: on OpenAI's MLE-Bench (75 real Kaggle competitions), AIDE with o1-preview earned medals in 16.9% of competitions — roughly four times the next-best agent scaffold at publication — and it outperformed human researchers on METR's RE-Bench in short-horizon settings. The company raised an $8M seed round (investors include BoxGroup, Founder Collective, and Golden Ventures) to build what it describes as recursively self-improving software, and its aideml repository has 1,300+ GitHub stars.

FAQ

What do I need to get started? A codebase, a command that evaluates it, and a metric to improve. Install the open-source CLI (pip install weco), point it at your script, and define the goal.

Is Weco free? The CLI and the aideml research agent are open source; the hosted platform meters agent usage, with pricing tied to compute and model calls.

How is this different from Copilot-style assistants? Assistants suggest code as you type. Weco runs closed-loop experiments — edit, evaluate, learn, repeat — and only keeps changes that measurably improve your metric.

Can I trust the changes it makes? Every candidate is validated by your own evaluation command, and the full experiment tree is inspectable, so nothing lands without a measured result.

tl;dr

Weco AI's AIDE agent autonomously rewrites and re-tests ML code — kernels, models, prompts — until your chosen metric improves, with benchmark-topping MLE-Bench results and an $8M seed behind it.

Related

Looking for more options? Browse the Developer Tools directory or read our best AI coding tools listicle. Weco AI is also tracked on Crunchbase.