Brown University researchers propose SwarmEvolve, an evolutionary-search accelerator that moves multi-agent reasoning out of verbose text CoT into latent space: each round runs a swarm of ~450 agents that exchange ideas via KV caches, then a synthesizer agent emits a program. On 21 math and scientific-computing tasks, the paper reports a median of only ~3–8 minutes to match other SOTA frameworks’ 1-hour scores, and Qwen3-8B improved AlphaEvolve constructions on 3 problems (arXiv:2611.08483).
Why drop text CoT?
LLM evolutionary search repeatedly generates, evaluates, filters, and rewrites math/program candidates. Diversity usually means many agents first dump long chain-of-thought text, which is painfully slow. SwarmEvolve asks whether you can skip that text while still mixing ideas—adjacent to our coverage of multi-agent formalization, but via latent communication instead of pure text chat.
Method in brief
It builds on latent reasoning (continuous vectors, no token-by-token spill) and latent messaging (swap internal states). Training-free LatentMAS already hands KV caches to a synthesizer; SwarmEvolve scales that into evolutionary search.
Each round ~450 latent agents form groups with different specialties (~40 latent steps each). A novelty filter keeps diverse trajectories by cosine distance on hidden states—no text conversion first. Group synthesizers read retained members’ KV caches and cross ideas from different parent programs in latent space.
Results and limits
Vs OpenEvolve, ShinkaEvolve, and CodeEvolve on Qwen3 8B/14B/32B-FP8 (1 hour, single GPU), SwarmEvolve leads mean score, evaluates ~5–13× more programs per hour, and hits others’ 1-hour quality in a ~3–8 minute median; its mean also beats three baselines that use Qwen3-32B-FP8. Swapping latent for text reasoning drops Qwen3-8B scores even with a larger text token budget—longer text slows each round. Bigger swarms help early coverage most; late scores converge across sizes, while stronger LLMs still lift finals. Takeaway: for problems with clear evaluators, cheap latent swarms can put more wall-clock into explore-and-test on one GPU.
Limits: latent + synthesizer agents must share one LLM (KV isn’t cross-model); design assumes standard attention (linear/SSM/hybrids need state merging); KV memory grows with agents, steps, and layers. Future work: KV translators, compressed latent messages, multi-GPU sharding. More Agent news: AI News.
FAQ
Q: What pain does SwarmEvolve target? A: Multi-agent evolutionary search that writes long text CoT before code is slow; it reasons and communicates in latent space, then synthesizes programs from KV caches.
Q: How much faster vs other evolutionary stacks? A: The paper cites a ~3–8 minute median to match other SOTA 1-hour scores on 21 tasks, with ~5–13× programs evaluated per hour.
Q: Main limitations? A: Same-model KV only, standard-attention dependence, and linear VRAM growth; cross-model collaboration remains open.
相关文章
