Open-Ended Frontiers

1. POET

POET (Wang, Lehman, Clune, Stanley 2019) — Paired Open-Ended Trailblazer — co-evolves a population of environments(parameterised obstacle courses) alongside a population of agents trying to solve them. Three crucial mechanisms:

• Minimal-criterion novelty: a new environment is admitted only if at least one agent can make some progress on it (avoiding trivially-impossible environments) and only if it is novel relative to existing environments.
• Cross-environment transfer: agents periodically migrate from one environment to another. An agent specialised on environment A may, after a small adaptation period, dramatically outperform a specialist on environment B — uncovering “goal-switching” as a source of novelty.
• Environment generation: environments are mutated and selected to provide intermediate-difficulty challenges for agents that already exist (an automatic curriculum).

POET produces increasingly complex obstacle courses and agent gaits that traverse them, neither of which the experimenter explicitly specified. The successor Enhanced POET (Wang et al. 2020) adds a learned environment-novelty metric and richer transfer logic.

2. AI-GAs

AI-Generating Algorithms (Clune 2019) is the proposed research programme: instead of hand-engineering AI, build a system that evolves AI — including its architecture, learning rules, and the learning environment in which it lives. The three pillars Clune identifies:

• Meta-learn architectures.
• Meta-learn learning algorithms.
• Generate effective learning environments / curricula.

POET addresses the third pillar; differentiable architecture search (NAS) addresses the first; learnable optimisers (L2L, learning-to-learn) address the second. The AI-GAs hypothesis is that combining all three will produce capabilities that no one programme can achieve alone.

3. Evolutionary Architecture Search

Architecture search is the most consequential industrial application of evolutionary methods in the deep era.

• AmoebaNet (Real et al., Google 2019): regularised evolution with tournament selection over a cell-search space defined by NASNet. Beat hand-designed architectures and reinforcement-learning-based NAS on ImageNet at the time.
• RegNet & pre-NAS regularised evolution (Radosavovic et al. 2020): factor architecture search through evolutionary methods over design spaces rather than discrete architectures. A more parsimonious description of what works.
• EvoPrompt & LLM-driven NAS(2023–2024): use LLMs to propose mutations of architecture descriptions (or whole programmes); evolutionary selection on the proposals. The natural successor to FunSearch, applied to architecture rather than mathematics.

4. Evolved Optimisers

Lion (Chen et al., Google 2023) was discovered by symbolic-regression-style program search over the space of optimiser update rules. The result is a single closed-form rule that beats AdamW on transformer-LM training:

\[ \mathbf{u} = \mathrm{sign}\bigl(\beta_1 \mathbf{m} + (1-\beta_1)\mathbf{g}\bigr),\quad \mathbf{w} \leftarrow \mathbf{w} - \alpha\bigl(\mathbf{u} + \lambda \mathbf{w}\bigr) \]

Discovered by an evolutionary program-search loop with deep learning as the inner evaluator. The first widely-deployed example of evolved code that improves a core ML tool. Subsequent work (Schmidhuber-style learning-to-learn fused with EA program search) pushes the frontier further: discovering loss functions, augmentation strategies, attention variants.

5. FunSearch and LLM + EA

FunSearch (Romera-Paredes et al., DeepMind 2024) is the prototype LLM + evolution system that has produced novel mathematics. The recipe:

1. Maintain a database of programs that score well on a target evaluator.
2. Sample a few high-scoring programs, prompt the LLM to write an “improved” version, run the evaluator on the proposal.
3. Insert the proposal into the database if it improves any cluster of similar programs.
4. Repeat for \(\sim 10^6\) generations.

FunSearch produced new improvements to the cap-set bound in extremal combinatorics and to online bin-packing — results published in Nature. The pattern generalises far beyond mathematics; we expect LLM-driven evolutionary loops to become a major mode of scientific computation through the late 2020s.

6. Open-Endedness as a Research Programme

The deepest question in this space: can we build a system whose long-run behaviour is like biological evolution — producing genuinely novel artefacts forever, with no upper limit, no convergence, no diminishing returns? Stanley, Lehman, Soros, and Sigaud have posed this as a programme rather than a single algorithm. The ingredients we believe to be necessary:

• Co-evolution of agents and challenges (POET).
• Stepping-stone-friendly selection (novelty/QD).
• An unbounded, generative environment (open-ended physics, open-ended language).
• Mechanisms by which a discovery in one part of the system creates new opportunities elsewhere — the “adjacent possible” (Kauffman).

No system to date is convincingly open-ended. The conjecture is that scaling POET-like architectures with LLM-rich agents and procedurally-generative environments will move us closer; whether this is enough is an empirical question that the next decade will resolve.

7. Course Endings

Evolutionary computation entered the deep-learning era as a curiosity and is leaving it as a tool of choice for problems where gradients lie about the global landscape: architecture search, RL with sparse rewards, protein design, quality-diversity illumination, automatic algorithm discovery. Its theoretical grounding (information-geometric optimisation, runtime analysis, schema theorems) has matured. Its industrial track record (NAS, AmoebaNet, OpenAI ES, FunSearch, Lion) is now considerable. The open question is no longer whether evolutionary methods belong in the modern ML practitioner’s toolkit but how often they should be reached for first.