Evolutionary algorithms vs. poker games

1. Evolutionary algorithmsvs.poker games Yikan Chen (yc2r@virginia.edu) Weikeng Qin (wq7yt@virginia.edu)

2. Outline Evolutionary Algorithm E-ANN Poker! Artificial Neural Network

3. Evolutionary algorithm

4. Evolution Process Evolutionary algorithm Crossover Mutation Natural Selection • Evolutionary Algorithm

5. Encoding and Crossover Evolutionary algorithm 1 1 1 0 0 1 1 0 0 1 0 0 1 0 1 1 0 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1

6. Mutation Evolutionary algorithm 1 1 1 0 0 1 1 0 1 1 0 0 0 1 1 1

7. Natural Selection Evolutionary algorithm Run the roulette-wheel selection based on the fitness value of candidates

8. Important Parameters • Crossover rate • Mutation rate • Elite rate • Fitness function • Demo http://userweb.elec.gla.ac.uk/y/yunli/ga_demo/ Evolutionary algorithm

9. AKQ 2-player game • $1 blinds for each player • Player1 bet or fold • Player2 call or fold Evolutionary algorithm & poker

10. Derive the optimal strategy using EA • Chromosomal representations • Fij: fold threshold when Pi got Cardj • Fitness functions Evolutionary algorithm & poker

11. Fitness functions • Fi: fitness function • Wij: money won by candidate I against candidate j Evolutionary algorithm & poker

14. Decreased fluctuation Further decreased fluctuation Evolutionary algorithm & poker

15. Real Texas Hold’em • Encoding Strategy (Turn and River) • Hand strength (player confidence) • Fraction of opponent raise (opponent confidence) • Total raise (profit) Evolutionary algorithm & poker

16. Fitness Criterion Evolutionary algorithm & poker

17. Performance Evolutionary algorithm & poker

18. Artificial neural network: review

19. Artificial neural network: review a1 w1 ∑ a2 w2 f output …… wn an b 1

20. Artificial neural network: review Evolving Topology Input output Hidden Layer

21. Simplest Encoding Method E-ANN (evolutionary ann) d c b a a b c d

22. http://www.cs.utexas.edu/users/nn/ • Neuro Evolution of Augmenting Topologies • Encoding Strategy: Node-based • Neuron gene table • Link gene table • Innovation number • Global database of innovations • Each innovation has unique ID number Neat e-ann

23. Neat e-ann

24. Mutation • Perturb weights • Add a link gene • Add a neuron gene • Crossover • By innovation number Neat e-ann

25. Crossover Neat e-ann 4 4 6 5 5 1 2 3 1 2 3 1 1->4 2 2->4 3 3->4 4 2->5 5 5->4 8 1->5 1 1->4 2 2->4 3 3->4 4 2->5 5 5->4 6 5->6 7 6->4 9 3>5 10 1->6

26. Crossover Neat e-ann 4 6 5 1 2 3 1 1->4 2 2->4 3 3->4 4 2->5 5 5->4 6 5->6 7 6->4 8 1->5 9 3>5 10 1->6

27. Simplified Poker Model • 1-10 • Initial credit: 10 chips • One chip ante at the beginning • Call, raise (1 chip each time), fold • Tournament E-ANN & poker

28. e-ann & poker Two player game

29. e-ann & poker

30. Four different types of opponents e-ann & poker Tight Aggressive (TA) Tight Passive (TP) Loose Aggressive (LP) Loose Passive (LP)

31. α: min win probability to call β: min win probability to raise e-ann & poker

32. e-ann & poker A: player type B: player action

33. e-ann & poker

34. Bluffing…… e-ann & poker

35. Thanks!