EvoNash: Accelerating Convergence to Nash Equilibrium

EvoNash investigates whether adaptive mutation helps populations of AI agents reach Nash Equilibrium faster than a fixed mutation rate.

In each experiment, 1,000 AI agents with neural network brains play the Iterated Prisoner's Dilemma, a classic game theory scenario where players must choose to cooperate or defect. For up to 1,500 generations, agents evolve through natural selection, where the best-performing strategies survive and reproduce with small random changes (mutations).

The control group uses a fixed 5% mutation rate, where every agent mutates the same amount regardless of performance. The experimental group uses adaptive mutation, where struggling agents mutate more to explore new strategies, and successful agents mutate less to preserve what works.

By running both groups from identical starting conditions using seed pairs, the experiment isolates mutation strategy as the only variable. Across 1,800+ experiments and ~320 billion strategic decisions simulated on distributed GPUs, the results show adaptive mutation converges to Nash Equilibrium significantly faster.

The idea for this science project started after I learned about the Prisoner's Dilemma, a Game Theory scenario where players choose to cooperate or betray each other. I’m also fascinated by Artificial Intelligence, so I thought it would be intriguing to combine the two concepts. I decided to investigate the impact of different mutation strategies on AI learning.

This question became EvoNash. I built a simulation where 1,000 AI agents with neural network brains play the Iterated Prisoner's Dilemma. Over generations, the best strategies survive and reproduce with small random mutations, just like biological evolution. I wanted to test the outcome when mutation changes are based on an agent's fitness score, rather than a standard static mutation rate. What would happen if poor performing agents mutated more, searching harder for better strategies, while successful agents mutated less to preserve their successful traits? Could populations using this strategy reach Nash Equilibrium faster, a stable state where no one benefits from changing their strategy.

My hypothesis was: If the mutation rate of a neural network is dynamically scaled inversely to its fitness score, then the population will converge to Nash equilibrium faster than populations using a fixed mutation rate.

This research benefits tech companies, scientists and the environment. Finding more efficient training methods makes AI greener and faster by saving energy. Additionally, teaching AI to quickly reach stable, cooperative strategies can be used to help solve other complex real-world problems from Autonomous vehicles and robotics, to Drug discovery and Cybersecurity.

Research:

First I started to learn about game theory. I discovered many interesting algorithms, including Nash's equilibrium concept, and the Prisoner's Dilemma. I explored how neural networks are trained, and how genetic algorithms use selection and mutation to solve problems. For references, I used credible sources that had citations.

Experimental Design:

I wrote my simulation in Python using PyTorch, a machine learning library that runs calculations on GPUs for speed. Each experiment creates 1,000 AI agents with small neural networks as brains. These agents play an iterated Prisoner's Dilemma against each other for 750 ticks per generation, deciding whether to cooperate or defect.

After each generation, agents are ranked by score. The top 20% of performers survive and reproduce with small random mutations to their neural networks, like DNA mutations in biology. Poor performers are replaced. This repeats for up to 1,500 generations or until the population converges to Nash Equilibrium. I ran matched pairs of experiments comparing a fixed mutation rate (control) against an adaptive mutation (experimental).

Controlling Variables:

Other than the mutation strategy, all factors were kept constant: population size, selection pressure, neural network architecture and simulation ticks. Most importantly, identical seed pairings were used. This ensured identical starting conditions: agent positions, neural network weights, and random choices.

Collecting Data:

I built a distributed computing system where multiple GPU-equipped computers pull experiment jobs from a central server. Each completed experiment uploads its generation-by-generation data to a database, and calculates the mean generations to reach convergence. A live web dashboard at https://sf.defouw.ca analyzes everything automatically and performs statistical tests like Welch's t-test and calculates effect sizes.

Sample Size:

I ran over 1,800 experiments across multiple random seeds, with paired control and experimental runs for each seed, totaling approximately 320 billion individual game decisions.

Key Findings

Policy entropy (a measure of how randomly agents choose their actions) across both experimental conditions followed an exponential decay pattern, dropping sharply in early generations before leveling off near zero. (Graph 1) This indicates that agents rapidly shifted from random decision-making to a single shared strategy consistent with Nash equilibrium.

My results showed that both the static (control) and adaptive (experimental) mutation groups achieved comparable final fitness scores, but the adaptive group converged to Nash equilibrium significantly faster. This shows that adaptive mutation accelerates strategy optimization without reducing agent performance. Across all seed pairs, the experimental group (adaptive mutation) consistently reached Nash Equilibrium faster than the control group (fixed mutation). (Graph 2) On average, adaptive mutation converged at generation 200+/-22, compared to generation 227+/-12 for the control. This is a 27-generation improvement, or about 12% faster. (Table 1)

This pattern wasn't from one lucky seed. When I examined individual seed pairs, where both groups started from identical conditions, the experimental group converged first across all pairs. This is visible in the Paired Seed Convergence graph and Convergence Generation by Seed Pair chart (Graph 3), where the purple bars (experimental) are consistently shorter than the blue bars (control) across all seeds.

Statistical Analysis

With over 1,500 converged experiments, I needed rigorous statistical methods to confirm the difference wasn't just by chance or random noise.

I used Welch's t-test, which compares the means of two groups while accounting for differences in sample size and variance between them. This was important because my control and experimental groups had different numbers of completed experiments and varying amounts of spread in their data. The test returned a p-value of 2.56e-131 (Table 1). This means if there were truly no difference between the groups, the chance of seeing results this extreme are nearly zero! In science, anything below 0.05 is considered statistically significant, and my result far exceeds that threshold.

Since I found a difference in data, I also calculated Cohen's d (Table 1), to measures how large the difference is. A value of 0.2 is considered small, 0.5 medium, and 0.8 large. My experiment produced a Cohen's d of 1.58 (a very large effect), confirming this isn't a subtle difference but a clear, meaningful advantage for adaptive mutation.

Finally, I verified statistical power, to determine whether I had enough data to trust the results. My experiment achieved 100% power (Table 2), meaning I had far more experiments than required to reliably detect this effect.

Summary of Findings

Across all metrics, the adaptive mutation group consistently reached Nash equilibrium faster than the static mutation group while maintaining comparable agent performance. These findings were shown to be statistically significant with a meaningful effect size.

Conclusion

My results support my hypothesis that adaptive mutation significantly accelerates convergence to Nash Equilibrium compared to a fixed mutation rate. With a p-value of 1.91e-133 and a large effect size (Cohen's d = −1.58), this finding is both statistically significant and meaningful.

Also faster convergence didn't sacrifice performance. Both groups achieved comparable peak fitness scores, meaning adaptive mutation found equally strong strategies in fewer generations.

What I Learned

I learned that a fixed mutation rate works, but it isn't optimal. Adapting the mutation rate based on fitness works better. Struggling agents get more room to experiment, while successful agents retain their strengths, allowing populations to reach stable strategies sooner.

This has implications beyond my simulation. In AI, adaptive learning rates are already used to train neural networks more efficiently. My results suggest this works not just as an engineering trick, but because it reflects a real advantage built into how evolution naturally occurs.

In economics and game theory, faster convergence means multi-agent systems can reach stable outcomes with less energy and computing power. In biology, it supports the theory of evolvability, the idea that organisms capable of regulating their own mutation rates have a selective advantage over those that cannot.

Finally, I learned that experimental design matters as much as the results themselves. Using seed pairs ensured both groups started under identical conditions, allowing me to prove the differences were caused by mutation strategy alone, and not by random chance.

Improvements:

I started by running 100s of experiments with a single seed, before realizing I needed multiple seed pairs to prove that the results weren't from a single start state. I had to rush to collect enough paired data across different seeds. Next time I would run less experiments per seed and do more seeds day one.

Future Work:

I would try bigger + more complex worlds (3D vs 2D, multiple food types, predator-prey, etc), larger populations or bigger neural networks, or different dynamic mutation strategies. I could introduce cooperation and/or communication between organisms to create richer game theory scenarios.

Acknowledgments

I would like to thank my dad, Matt deFouw, who provided the server equipment and GPU-equipped computers that made this experiment possible. Running over 1,800 experiments required significant computing power, and he helped set up the distributed computing infrastructure and offered guidance on programming challenges throughout development. His technical support turned an idea into a working system.

I also want to thank my mom, Tarlene deFouw, who helped with the statistical analysis and scientific rigor of the project. She guided me in selecting appropriate statistical tests, understanding p-values and effect sizes, and ensuring my experimental design met the standards of a properly controlled study. Her input strengthened the credibility of my results.

I am very grateful for their ongoing support and encouragement.

28 sources across 9 categories

Game Theory & Nash Equilibrium

1

Nash, J.F. (1950). Equilibrium points in n-person games. Proceedings of the National Academy of Sciences, 36(1), 48–49 doi:10.1073/pnas.36.1.48

2

Nash, J.F. (1951). Non-cooperative games. Annals of Mathematics, 54(2), 286–295 doi:10.2307/1969529

3

von Neumann, J. & Morgenstern, O. (1944). Theory of Games and Economic Behavior. Princeton University Press

4

Maynard Smith, J. (1982). Evolution and the Theory of Games. Cambridge University Press doi:10.1017/CBO9780511806292

5

Axelrod, R. (1984). The Evolution of Cooperation. Basic Books

Neural Networks

6

McCulloch, W.S. & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5(4), 115–133 doi:10.1007/BF02478259

7

Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386–408 doi:10.1037/h0042519

8

Rumelhart, D.E., Hinton, G.E. & Williams, R.J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533–536 doi:10.1038/323533a0

9

Goodfellow, I., Bengio, Y. & Courville, A. (2016). Deep Learning. MIT Press [Link]

Evolutionary Computing & Neuroevolution

10

Holland, J.H. (1975). Adaptation in Natural and Artificial Systems. University of Michigan Press

11

Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley

12

Stanley, K.O. & Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies. Evolutionary Computation, 10(2), 99–127 doi:10.1162/106365602320169811

13

Salimans, T., Ho, J., Chen, X., Sidor, S. & Sutskever, I. (2017). Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864 [Link]

14

Such, F.P., Madhavan, V., Conti, E., Lehman, J., Stanley, K.O. & Clune, J. (2017). Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv preprint arXiv:1712.06567 [Link]

Adaptive Mutation & Self-Adaptation

15

Bäck, T. (1993). Optimal mutation rates in genetic search. Proceedings of the 5th International Conference on Genetic Algorithms, 2–8

16

Smith, J.E. & Fogarty, T.C. (1997). Operator and parameter adaptation in genetic algorithms. Soft Computing, 1(2), 81–87 doi:10.1007/s005000050009

17

Eiben, A.E., Hinterding, R. & Michalewicz, Z. (1999). Parameter control in evolutionary algorithms. IEEE Transactions on Evolutionary Computation, 3(2), 124–141 doi:10.1109/4235.771166

18

Karafotias, G., Hoogendoorn, M. & Eiben, A.E. (2015). Parameter control in evolutionary algorithms: Trends and challenges. IEEE Transactions on Evolutionary Computation, 19(2), 167–187 doi:10.1109/TEVC.2014.2308294

Stress-Induced Mutagenesis

19

Radman, M. (1975). SOS repair hypothesis: Phenomenology of an inducible DNA repair which is accompanied by mutagenesis. Basic Life Sciences, 5A, 355–367 doi:10.1007/978-1-4684-2895-7_48

20

Tenaillon, O., Taddei, F., Radman, M. & Matic, I. (2004). Second-order selection in bacterial evolution: Selection acting on mutation and recombination rates in the course of adaptation. Research in Microbiology, 155(6), 457–463 doi:10.1016/j.resmic.2004.01.013

Statistical Methods

21

Welch, B.L. (1947). The generalization of 'Student's' problem when several different population variances are involved. Biometrika, 34(1–2), 28–35 doi:10.1093/biomet/34.1-2.28

22

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates

23

Hedges, L.V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128 doi:10.3102/10769986006002107

24

Mann, H.B. & Whitney, D.R. (1947). On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics, 18(1), 50–60 doi:10.1214/aoms/1177730491

Multi-Agent Systems & Reinforcement Learning

25

Shoham, Y. & Leyton-Brown, K. (2009). Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press

26

Sutton, R.S. & Barto, A.G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT Press [Link]

GPU Computing

27

Nickolls, J., Buck, I., Garland, M. & Skadron, K. (2008). Scalable parallel programming with CUDA. ACM Queue, 6(2), 40–53 doi:10.1145/1365490.1365500

Information Theory

28

Shannon, C.E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423 doi:10.1002/j.1538-7305.1948.tb01338.x