RECONNAISSANCE BLIND CHESS

Prizes


We offer a symbolic prize of $1,000 to the first-place winner (excluding any baseline bots from the rankings).

A limited number of competitors (approximately 5) will be given access to NeurIPS tickets based on the performance of their bots along with the techniques they employed. At minimum, bots ranked in the top 3 places (including baseline bots in the rankings) will be given access to tickets. We will also seek to give ticket access to one or more additional bots with promising or interesting approaches that may not have performed especially well yet, weighing bot performance with the approach used. (Following the tournament, we will request 1-paragraph method-descriptions from those interested in having access to a ticket to participate in NeurIPS and determine awardees.) Competitors awarded access to a ticket will also get to share their approach during the competition session at NeurIPS. Note that we can only award access to 1 ticket per bot, and the person granted ticket access must be one of the people speaking about the bot during the competition session (per NeurIPS rules). (This prize does not cover the cost of the tickets or any cost to attend NeurIPS, which would still need to be purchased.)

Post Tournament Approach Sharing


In order to build collective knowledge following the tournament:

  • We expect developers of the top performing bots (those receiving ticket access and potentially others) will present their approach during the competition session (Friday, December 13th - Saturday, December 14th) at the NeurIPS, or, if unable to attend, find someone who can present their approach. (In this case, we will provide the presenter with access to a NeurIPS ticket if needed.) As a last resort, developers may work with us to ensure that an appropriate presentation can be given.

  • We highly encourage successful participants (including those for baseline algorithms) to share their approaches in open venues of their choice after the competition.

Rules


Players' bots will play games by directly interacting with our server, which will manage the games. I.e., each participant runs her bot remotely. Competitors do not need to attend NeurIPS although attendance is highly encouraged, and we expect successful participants who cannot attend to work with us to ensure that an appropriate presentation can be given.

  • We try to make the rules flexible, but ask that participants are honest with any information they share after the tournament.

  • Participants are allowed to use any available libraries or software (including chess libraries like Stockfish, for example) as part of their bot.

  • Participants may use as much of the provided code as they like but are not required to use any of it.

  • There are no restrictions on the computational resources employed by participants' bots.

  • Each bot is required to support playing at least 4 simultaneous games. (This requirement may change as we learn more about the number of participants.) (Our sample bot code and python API handles the simultaneous game support fairly transparently for the user with an adjustable parameter on how many simultaneous games are allowed, which is set to 4 by default.)

  • Participants may not share their private bot key with anyone.

  • Although this is not the intent of the competition, human intervention is allowed as are live modifications to one's bot; again, we ask that participants are honest about what they do, however.

  • Networking issues are the responsibility of the participant. (However, while we make no promises about what we will be able to do, if it seems clear that a network issue alters top rankings, the organizers maintain discretion to try to resolve the issue in a fair manner. We do not want uncontrollable networking issues significantly affecting the tournament).

  • When no bot is connected for a scheduled game and the game times out, both players get a loss.

  • By participating in the competition, competitors agree to allow the organizers to use the collected data (moves and timing) in any manner. All game replays will be available on our website.

Participant and Bot Eligibility


  • Unless explicitly stated otherwise, everyone age 18 years of age or older is eligible to compete for the monetary prize and the awarded access to NeurIPS tickets.

  • Competition organizers and members of the Johns Hopkins University Applied Physics Laboratory (JHU/APL), the leading organization, cannot compete for the monetary prize or access to the NeurIPS tickets that are awarded as prizes.

  • Competition organizers and members of JHU/APL may create baseline algorithms (indicated with an adjacent "Baseline" icon), which may play against other bots in the final tournament.

  • Others from organizing institutions, except JHU/APL, may compete for the monetary prize.

Tournament Protocol and Ranking


In the final tournament, each bot will play 24 games against every other bot in the tournament, 12 as black and 12 as white. Bots will be ranked simply based on the number of games they win. (The bot with the most wins is the first place winner and so on.) Any ties will be broken by considering the specific record of the tied bots when they played against each other. (E.g., if bot A and bot B both win 242 games in the tournament, but bot A won 13 and lost 11 games against bot B, bot A would win the higher ranking.) Remaining ties will be broken by subsequent series of 40 games between the tied bots (20 as white and 20 as black) until the tie is broken.

We use number of wins rather than other ratings like BayesELO because the metric is transparent and order independent, and we are able to let every bot play every other bot a significant number of times, which largely obviates the need for more advanced rating systems.

Logistics


Players' bots will play games by interacting with our server either indirectly, through our provided python libraries, or directly, via the RESTful API. Prior to the time of a tournament, participants are expected to have their bots up and running, at which time, they would be querying the server for assigned games. When the tournament begins, the server will inform the bots of their current games, and the bots play by sending actions to and receiving results from the server. The server will enable viewers to observe past games. It will similarly maintain a real-time leader board Teams will be required to support continuously playing at least 4 games simultaneously (the mechanics for which are provided by our code) for up to 1.5 weeks. This number may change if we determine it to be necessary.

Upon registration, each player will set a password, which will be used to authenticate that player's messages to the server. (We do not address more advanced networking or computer attacks such as someone sniffing traffic of another player.)

Test Tournaments


We have two test tournaments (starting August 13th and September 9th) to allow participants to ensure that they are properly interfacing with the server and to ensure that our server can support all the participants. Participation in the test tournaments is optional. The test tournaments have no bearing on the final rankings and participants are welcome to submit any (poor- or good-performing) bot. To help us ensure the fidelity of our infrastructure, we ask that the bots move at roughly the rate of the one they expect to use in the final tournament or faster.