Tuesday, July 29, 2008

2008 Poker Bot Competition Summary

2008 Poker Bot Competition Summary

The 2008 poker competition consisted of 12 competitors from 6 countries and 19 programs. The results were announced at AAAI 2008 on July 15, 2008 in Chicago, IL.

Heads-Up Texas Hold'em Limit

Competitors

Program NameAffiliationTeam Members
Dr. Sahbak(No affiliation)Roy Fox
GUSGeorgia State UniversityNicholas Larsen
GS4-BetaCarnegie Mellon UniversityAndrew Gilpin, Tuomas Sandholm, Troels Bjerre Sørensen
Hyperborean08-Online and Hyperborean08-EquilibriumUniversity of AlbertaMichael Bowling, Duane Szafron, Nolan Bard, Neil Burch, John Hawkin, Michael Johanson, Morgan Kan, Nick Abou Risk, Dave Schnizlein, Kevin Waugh, Martin Zinkevich, Bryce Paradis
Ian FellowsUniversity of California, San DiegoIan Fellows
GGValutaUniversity of BucharestMihai Ciucu, Stefan Popescu, Mihaita Alexandru Leoveanu, Diana Dulgheru
PokeMinnLimit1 and PokeMinnLimit2University of MinnesotaMohamed Elidrisi, Brett Borghetti

Results

The Heads-Up Limit competition features two winner determination rules. The bankroll rule orders the competitors according to the total number of chips won. The wins rule orders the competitors according to their wins and losses against the other competitors, and uses the instant runoff technique (see the rules page for further information).

60,000 hands were played for each pair of competitors, for a total of 2,160,000 hands. The following table summarizes the pairwise and overall performance for each entry.


GS4-Beta Hyperborean08-Online Hyperborean08-Equilibrium Ian Fellows GGValuta PokeMinnLimit2 PokeMinnLimit1 GUS Dr. Sahbak Average
GS4-Beta
-0.03 ± 0.012 -0.029 ± 0.009 -0.02 ± 0.015 -0.036 ± 0.009 0.2 ± 0.018 0.113 ± 0.016 0.695 ± 0.018 0.713 ± 0.007 0.201 ± 0.007
Hyperborean08-Online0.03 ± 0.012
-0.002 ± 0.014 0.004 ± 0.005 -0.007 ± 0.012 0.202 ± 0.017 0.184 ± 0.01 0.52 ± 0.009 0.616 ± 0.01 0.194 ± 0.006
Hyperborean08-Equilibrium 0.029 ± 0.009 0.002 ± 0.014
0.024 ± 0.003 0.005 ± 0.004 0.182 ± 0.008 0.161 ± 0.02 0.476 ± 0.019 0.566 ± 0.013 0.181 ± 0.004
Ian Fellows 0.02 ± 0.015 -0.004 ± 0.005 -0.024 ± 0.003
0.003 ± 0.004 0.153 ± 0.017 0.154 ± 0.02 0.467 ± 0.012 0.532 ± 0.019 0.163 ± 0.005
GGValuta 0.036 ± 0.009 0.007 ± 0.012 -0.005 ± 0.004 -0.003 ± 0.004
0.144 ± 0.013 0.143 ± 0.008 0.452 ± 0.017 0.494 ± 0.094 0.159 ± 0.011
PokeMinnLimit2 -0.2 ± 0.018 -0.202 ± 0.017 -0.182 ± 0.008 -0.153 ± 0.017 -0.144 ± 0.013
0.03 ± 0.021 0.743 ± 0.02 1.265 ± 0.017 0.145 ± 0.008
PokeMinnLimit1 -0.113 ± 0.016 -0.184 ± 0.01 -0.161 ± 0.02 -0.154 ± 0.02 -0.143 ± 0.008 -0.03 ± 0.021
0.299 ± 0.014 0.921 ± 0.02 0.054 ± 0.011
GUS -0.695 ± 0.018 -0.52 ± 0.009 -0.476 ± 0.019 -0.467 ± 0.012 -0.452 ± 0.017 -0.743 ± 0.02 -0.299 ± 0.014
0.512 ± 0.014 -0.393 ± 0.007
Dr. Sahbak -0.713 ± 0.007 -0.616 ± 0.01 -0.566 ± 0.013 -0.532 ± 0.019 -0.494 ± 0.094 -1.265 ± 0.017 -0.921 ± 0.02 -0.512 ± 0.014
-0.702 ± 0.01

Heads-Up Limit Bankroll

The number in parentheses is the average number of small bets won per hand.
  1. GS4-Beta (0.201)
  2. Hyperborean08-Online (0.194)
  3. Hyperborean08-Equilibrium (0.181)
  4. Ian Fellows (0.163)
  5. GGValuta (0.159)
  6. PokeMinnLimit2 (0.145)
  7. PokeMinnLimit1 (0.054)
  8. GUS (-0.393)
  9. Dr. Sahbak (-0.702)

Heads-Up Limit Wins

The numbers in parentheses are the overall number of wins and losses. Note that there is a three-way tie for second.
  1. Hyperborean08-Equilibrium (8-0)
  2. Hyperborean08-Online (6-2)
  3. Ian Fellows (6-2)
  4. GGValuta (6-2)
  5. GS4-Beta (4-4)
  6. PokeMinnLimit2 (3-5)
  7. PokeMinnLimit1 (2-6)
  8. GUS (1-7)
  9. Dr. Sahbak (0-8)

Heads-Up Texas Hold'em No Limit

100,000 hands were played for each pair of competitors for a total of 600,000 hands.

Competitors

Program NameAffiliationTeam Members
University of BallaratUniversity of BallaratPeter Vamplew, Chris Turville, Robert Layton
BluffBot 3.0(No affiliation)Teppo Salonen
Tartanian2-BetaCarnegie Mellon UniversityAndrew Gilpin, Tuomas Sandholm, Troels Bjerre Sørensen
Hyperborean08University of AlbertaMichael Bowling, Duane Szafron, Nolan Bard, Neil Burch, John Hawkin, Michael Johanson, Morgan Kan, Nick Abou Risk, Dave Schnizlein, Kevin Waugh, Martin Zinkevich, Bryce Paradis

Results

The following tables summarize the pairwise and overall performance of the bots. Since the winner determination rule was instant runoff bankroll, the programs did not finish in the same order as they would have if they were ordered by total bankroll. The tables illustrate the winner determination process.

Tartanian2-Beta BluffBot 3.0 Hyperborean08 Ballarat Average
Tartanian2-Beta
-0.611 ± 0.091 -0.625 ± 0.091 5.371 ± 0.223 1.378 ± 0.091
BluffBot 3.0 0.611 ± 0.091
-0.109 ± 0.046 2.566 ± 0.109 1.023 ± 0.052
Hyperborean08 0.625 ± 0.091 0.109 ± 0.046
2.13 ± 0.123 0.954 ± 0.054
Ballarat -5.371 ± 0.223 -2.566 ± 0.109 -2.13 ± 0.123
-3.355 ± 0.107

Hyperborean08 BluffBot 3.0 Tartanian2-Beta Average
Hyperborean08
0.109 ± 0.046 0.625 ± 0.091 0.367 ± 0.054
BluffBot 3.0 -0.109 ± 0.046
0.611 ± 0.091 0.251 ± 0.051
Tartanian2-Beta -0.625 ± 0.091 -0.611 ± 0.091
-0.618 ± 0.061

Hyperborean08 BluffBot 3.0 Average
Hyperborean08
0.109 ± 0.046 0.109 ± 0.046
BluffBot 3.0 -0.109 ± 0.046
-0.109 ± 0.046

Final Rankings

The number in parentheses is the average number of small bets won per hand.
  1. Hyperborean08-NoLimit (0.954)
  2. BluffBot 3.0 (1.023)
  3. Tartanian2-Beta (1.378)
  4. Ballarat (-3.355)

6-Player Texas Hold'em Limit

Competitors

Program NameAffiliationTeam Members
AKI-RealBot and mcBotUltraTechnical University DarmstadtJohannes F�rnkranz, Ulf Lorenz, Frederik Janssen, Sang-Hyeun Park, Eneldo Loza Menc�a, Jan-Nikolas Sulzmann, Oliver Uwira, Hendrik Schaffer, Thomas G�rge, Lars Meyer, Timo Bozsolik, Marian Wieczorek, Michael W�chter, Bj�rn Heidenreich, Andreas Eismann, Michael Herrmann, Arno Mittelbach, Bastian Christoph, Benjamin Herbert, Claudio Weck, Stefan L�ck, Immanuel Schweizer, Alexander Marinc, Kamill Panitzek
GUSGeorgia State UniversityNicholas Larsen
CMURing-BetaCarnegie Mellon UniversitySam Ganzfried, Andrew Gilpin, Tuomas Sandholm
Poki0University of AlbertaAaron Davidson, Darse Billings, Jonathan Schaeffer, Duane Szafron, Lourdes Pena, Nick Abou Risk, John Hawkin
DCUDublin City UniversityDavid Sinclair, Neill Sweeney

Results

The number in parentheses is the average number of small blinds won per hand.
  1. Poki0 (0.646)
  2. AKI-RealBot (0.573)
  3. DCU (0.252)
  4. CMURing-Prototype (0.163)
  5. mcBotUltra (-0.135)
  6. GUS (-1.50)