§ SEARCH & OBJECTIVE
How do you get a fly to walk when you can't write the gait and can't differentiate the physics? Define one number for “good” — forward distance — and let a gradient-free search climb it. It tuned 660 parameters from a stagger to a 86.6 mm walk.
What we tried, and how it went
It worked. Starting from random weights, a gradient-free search tuned the 660-parameter neural cellular automaton from a twitching stagger into a clean forward walk — 86.6 mm in 3 seconds, about 29 mm/s. No gradients, no hand-coded gait: score a candidate controller by how far the fly walks, keep what works, repeat.
What this isn't: this is the gait-controller search, and it succeeded. It's a separate experiment from the navigation reinforcement-learning attempt — that one tried to learn goal-directed steering and did not generalize. The locomotion search on this page is the one that produced a working walk.
§ A · WHY SEARCH, NOT GRADIENTS
You can't write the gait by hand, and you can't train it by backprop either. The fitness comes out of a contact-rich physics rollout — so the only thing you can do is try a controller and measure how far the fly walked.
§ B · WHAT COUNTS AS A GOOD WALK
“Good” has to become a single number the search can climb. Here it's the simplest one that turns “walk forward” into a score: distance, minus a small guardrail against collapsing.
Reward forward distance walked; subtract 0.05 for every control step the thorax sags below half its standing height ( = number of such steps, where ). Distance is measured after a short warm-up, so it scores control, not the initial settle.
the objective — one hand-set number, term by term
At the champion: F = 86.6 − 0.00 = 86.6 mm. Because the penalty stayed silent, the search was really just maximizing distance walked. It's a choice, not a law — distance alone happened to select a working gait; energy, symmetry, or speed-matching would each have picked a different one.
§ C · THE CLIMB — IT WORKED
Fifty generations of that search, scored by real MuJoCo rollouts. The best controller climbs from a stagger at F ≈ 0 to a clean walk at 86.6 mm — and that single climbing line is the gait getting better.
§ D · WATCH IT IMPROVE
The same three milestones — ① ② ③ from the climb — as side-by-side clips, so you could watch the gait go from stagger to walk. This footage has to be re-rendered from the checkpoints; the slots stand honest and empty until it is.
The same fly, three points along its own evolution. Each clip replays that generation's champion controller — reloaded from its checkpoint and rolled out through one shared world-fixed camera at the same scale and frame count — so the only thing changing is how far it gets. Watch it go stagger → lurch → clean walk: that growing displacement is the fitness climbing above it.
under the hood how a gradient-free search moves with only scores
The search is CMA-ES (population 32, σ₀ = 0.3). Given only scores, it samples a cloud of candidate controllers, keeps the best, and reshapes itself toward what worked — discovering which directions matter without ever seeing a gradient. One round, frozen to a picture:
- Sample ~32 candidate controllers from the current guess — the ellipse.
- Score each by how far its fly walked (one full rollout per candidate).
- Keep the best handful (green) and recenter the search on them; the ellipse contracts and reshapes toward what worked.
- Repeat. Fifty rounds of this is the climb above — score and rank, never a gradient.
What we chose
Two choices, one tab. The objective: forward thorax distance, minus a small stability penalty (0.05per control step the thorax sags below half its standing height) — the simplest scalar that turns “walk forward” into a number an optimizer can climb. The search: CMA-ES (gradient-free evolution strategy) over the ~660-parameter update rule — population 32, σ₀ = 0.3, 50 generations. It adapts a full covariance over the search space, so it discovers which directions matter without ever seeing a gradient — best reached F ≈ 86.6 mm.
Why
The objective is the simplest signal that produces walking at all: distance rewards locomotion, the penalty discourages collapsing or dragging (though in this run it never fired). And the search has to be gradient-free: the score comes from a contact-rich MuJoCo rollout — collisions, friction, stiff contacts — so F(θ) is not differentiable and backprop is off the table. CMA-ES only needs to score candidates, never differentiate them. The price is sample cost: every candidate is a full 3-second physics rollout, which is why the run is precomputed, not live.
Alternatives
Other scalars define “good” just as well — energy efficiency (distance per unit actuation), gait symmetry, speed-matching a target velocity, robustness to pushes, uprightness — and each would have selected a different controller. So would other searches: reinforcement learning (the route the whole-body fly papers take, a learned policy from reward), differentiable physics (MuJoCo MJX / Brax make the sim differentiable so you can backprop through contacts), or MAP-Elites for an archive instead of a single optimum. Comparing objectives needs fresh MuJoCo rollouts, so this tab visualizes the real one and namesthe rest; it doesn’t re-optimize.
The biological / frontier version
A single scalar optimized to a single winner is a strong assumption. Real behavior trades off many goals at once, and an animal carries a repertoire, not a maximum — so the frontier is multi-objective / quality-diversity search (MAP-Elites): keep an archiveof gaits that are each best at something — fast, stable, low-energy, tripod vs. wave — rather than one champion of one number. That’s Stage 3 of the campaign.
The math behind this — equations and constants — is in the appendix.