By darse billings dennis papp jonathan schaeffer duane szafron
1 / 24

Poker as a Testbed for Machine Intelligence Research - PowerPoint PPT Presentation

  • Uploaded on

By Darse Billings, Dennis Papp, Jonathan Schaeffer, Duane Szafron. Poker as a Testbed for Machine Intelligence Research. Presented By:- Debraj Manna Gada Kekin Dhiraj Raunak Pillani. CONTENT. Introduction Characteristics of Poker Game Texas Hold’Em Requirements From Players Lokibot

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Poker as a Testbed for Machine Intelligence Research' - sandra_john

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
By darse billings dennis papp jonathan schaeffer duane szafron l.jpg

By Darse Billings, Dennis Papp, Jonathan Schaeffer, Duane Szafron

Poker as a Testbed for Machine Intelligence Research

Presented By:-

Debraj Manna

Gada Kekin Dhiraj

Raunak Pillani

Content l.jpg

  • Introduction

    • Characteristics of Poker Game

    • Texas Hold’Em

    • Requirements From Players

    • Lokibot

    • Experiment

    • Future Work

Introduction l.jpg

  • Game Researchers used Chess & other board games as TestBed

  • Poker can be a better testbed for decision making problems

Poker l.jpg
POKER Szafron

  • Game of Imperfect knowledge

    • Risk management

    • Agent modelling

    • Unreliable information

    • Deception

  • Heuristic Search and evaluation methods employed in Chess not helpful.

Texas hold em l.jpg

  • Pre-Flop – Each player is dealt with two cards with their face down

  • Community Cards are dealt in 3 stages:-

    • Flop – 3 cards are dealt with face up

    • Turn – 4th community card is dealt with face up.

    • River – last community card is dealt

      • A round of betting held at each stage

  • Showdown – player having the best 5 cards wins the game

Betting strategy l.jpg

  • FOLD – Withdraw from the game

  • CALL – Match the current bet

  • RAISE – Raise the current outstanding bet

  • Only 3 raises are allowed in a round.

Requirement l.jpg

  • Hand Strength – strength of your hand compared to opponents.

  • Hand Potential – Probability of hand improving as additional cards appear.

  • Betting Strategy –Determining optimal betting strategy

  • Bluffing – Allows you make profit even on weak hands

Requirement contd l.jpg
REQUIREMENT (contd.) Szafron‏

  • Opponent Modeling – Determining probability distribution for opponents strategy.

  • Unpredictability – making difficult for opponent to model your strategy.

Slide10 l.jpg

Lokibot Szafron(later changed to Pokibot)‏

Slide11 l.jpg

Pre-flop Evaluation Szafron

  • 52 choose 2 = 1326 possible combinations for two cards

  • Approximate income rate for each starting hand using a simulation of 1,000,000 poker games done against nine random opponents

    • Highest income rate: A pair of aces

    • Lowest income rate: 2 and 7 (of different suits)‏

  • One time evaluation

Slide12 l.jpg

Hand Evaluation Szafron

  • Hand Strength

    • Assessment of the current strength of the hand

    • Enumeration techniques can provide an accurate estimate of the probability of currently holding the strongest hand.

  • Hand Potential

    • Potential changes in hand strength

Slide13 l.jpg

Hand Strength Szafron

  • Starting hand is and the flop is

  • 47 remaining unknown cards and {47 choose 2} = 1,081 possible hands an opponent might hold.

  • Hand strength is estimated by simply counting number of possible hands that are:

    • better than ours (any pair, two pair, A-K, or three of a kind: 444 hands)

    • equal to ours (9 possible remaining A-Q combinations)

    • worse than ours (628)‏

Slide14 l.jpg

Hand Potential Szafron

  • Hand strength alone is insufficient to assess the quality of a hand

  • Example

    • Hand:

    • Flop:

    • Next card: ,

  • Positive / Negative Potential

Slide16 l.jpg

Hand Potential (contd.) Szafron‏

  • If T{row,col} refers to the values in the table (B, T, A, and S are Behind, Tied, Ahead, and Sum, resp.) then Ppot and Npot are calculated by:

    Ppot = (T{B,A} + T{B,T}/2 + T{T,A}/2 ) / ( T{B,S} + T{T,S}/2)‏

    Npot = (T{A,B} + T{A,T}/2 + T{T,B}/2 ) / ( T{A,S} + T{T,S}/2)‏

  • Ppot = 0.208 and Npot = 0.274

Slide17 l.jpg

Betting Strategy Szafron

  • Hand strength and potential are combined into effective hand strength (EHS):

    EHS = HSn + (1 - HSn ) x Ppot

    where HSn is the adjusted hand strength for n opponents, Ppot is the positive potential.

  • EHS is the probability that we are ahead, and in those cases where we are behind there is a Ppot chance that we will pull ahead

    pot_odds = bets_to_us / ( bets_in_pot + bets_to_us )‏

  • Call when Ppot > pot_odds

Slide18 l.jpg

Experiment Szafron

  • Player A is the most advanced version of the program

  • Player E is a basic player

  • Player B lacks an appropriate weighting of subcases, using a uniform distribution for all possible opponent hands.

  • Player C uses a simplistic pre-flop hand selection method, rather than the advanced system which accounts for player position and number of opponents.

  • Player D lacks the computation of hand potential, which is used in modifying the effective hand strength and calling with proper pot odds.

Experiment contd l.jpg
Experiment (contd.) Szafron‏

  • The Bot was also run against other Poker playing bots and human players over the internet.

  • In it's current state the bot showed losses against advanced players

Slide21 l.jpg

Work In Progress Szafron

  • It is a predictable player that reacts the same in a given situation irrespective of any historical information

  • Opponent modeling: When Lokibot is better able to infer likely holdings for the opponent, it will be capable of much better decisions

  • Betting strategy: bluff with high potential hands and occasionally bet a strong hand weakly

Slide22 l.jpg

Work Done After The Paper Szafron

  • Later versions used simulation to discover the correct action to take, simulating what the actions of the other players (estimated using the opponent modelling) would be depending on the action that Lokibot chose.

  • They included selective sampling simulation: Opponent modelling consisted of weights for each hole card combination describing the probabilities of each action (bet, call, fold) and they measured opponents by their rate of each action.

  • The most recent work has concerned other approaches to poker game-tree search methods, as well as ways to evaluate perfomance of agents

Slide23 l.jpg

Contributions Of This Paper Szafron

  • Showing that poker can be a testbed of real-world decision making,

  • Identifying the major requirements of high-performance poker,

  • Presenting new enumeration techniques for hand-strength and potential, and

  • Demonstrating a working program that successfully plays "real" poker.

Reference l.jpg

  • Billings D., Papp D., Schaeffer J. and Szafron D. "Poker as a Testbed for Machine Intelligence Research." In Advances in Artificial Intelligence (Mercer R. and Neufeld E. eds.), Springer-Verlag, pp 1-15, 1998.