FedCSIS 2025 Challenge: Predicting Chess Puzzle Difficulty; submission deadline

This is the second edition of chess puzzle competition, this time with much bigger datasets and new data fields. The goal is to build a model to predict the difficulty (measured as Lichess rating) of given chess puzzles. The top 3 solutions will be awarded prizes (see, below).

Competition is organized within the framework of FedCSIS 2025 (see: www.fedcsis.org)

Overview

Main changes from the first edition:

• For every puzzle in both datasets we will now provide 22 success probabilitity predictions in both datasets. These are precomputed using chess engines and represent the predicted success chance of a player of given rating divided by 11 levels and two rating types (rapid and blitz). This change is meant to lower the entry bar for contestants without access to specialized hardware.
• Different data and bigger datasets, this time the training dataset has over 4.5 million instances, compared to 3.7 million in the first edition.

In a chess puzzle, the player assumes a role of White or Black in a particular configuration of pieces on a chessboard. The goal for the puzzle taker is to find the best sequence of moves, either outright checkmating the opponent or obtaining a winning material advantage.

On the Internet, chess puzzles are often found on chess websites, like Lichess. The moves from the opposing side are made automatically and the puzzle taker is provided with immediate feedback.

Solving puzzles is considered one of the primary ways to hone chess skills. However, currently the only way to reliably estimate puzzle difficulty is to present it to a wide variety of chess players and see if they manage to solve it.

The goal of the contest is to predict how difficult a chess puzzle is from the initial position of the pieces and moves in the solution. Puzzle difficulty is measured by its Glicko-2 rating calibrated by Lichess. In simplified terms, it means that Lichess treats each attempt at solving a puzzle like a match between the user and the puzzle. If the user solves the puzzle correctly, that counts as a win for the user and they gain puzzle rating while the puzzle loses rating. When the user fails to solve the puzzle, that counts as a loss and the opposite happens. Both user and puzzle ratings are initialized at 1500.

Each chess puzzle is described by the initial position (in Forsyth–Edwards Notation, or FEN) and the moves included in the puzzle solution (in Portable Game Notation, or PGN). The solution starts with one move leading to the puzzle position and includes both moves that the puzzle taker has to find and moves by the simulated opponent.

Authors of the top-ranked solutions (based on the final evaluation scores) will be awarded prizes funded by the Sponsors:

- 1000 USD for the winning solution + one FedCSIS 2025 registration
- 500 USD for the 2nd place solution + one FedCSIS 2025 registration
- 250 USD for the 3rd place solution + one FedCSIS 2025 registration

MORE INFO / CONTACT / ENROLLMENT:
https://knowledgepit.ml/predicting-chess-puzzle-difficulty-2/