Researchers from The University of Hong Kong, Shanghai Jiaotong University, Huawei Noah’s Ark Lab, and Shanghai AI Laboratory have introduced a new framework called DIFFUSEARCH. This innovative approach aims to improve decision-making in chess by moving away from traditional search algorithms like Monte Carlo Tree Search (MCTS).
Large language models (LLMs) often struggle with tasks that require multi-step reasoning, such as structured writing or problem-solving. This limitation affects their ability to make coherent decisions in complex scenarios. While some methods try to evaluate multiple options before making a choice, they can be costly in terms of computation and are prone to errors if predictions about the future are wrong.
MCTS and similar algorithms rely on repeated simulations to make decisions. However, they can become inefficient as they require significant computational resources. Furthermore, if the underlying value model is flawed, any errors can accumulate and lead to poor decision-making. This is especially problematic in tasks that require long-term planning.
To address these issues, the researchers developed DIFFUSEARCH, which uses discrete diffusion models. This method eliminates the need for costly search processes by training a model to predict future representations directly. By combining the world model and the policy into a single framework, DIFFUSEARCH reduces computational demands while enhancing efficiency and accuracy in long-term planning.
The researchers trained their model using supervised learning, with Stockfish, a well-known chess engine, providing labels for various board states. They explored different future representations and chose the action-state (s-asa) method for its simplicity. Instead of predicting sequences directly, the model employs discrete diffusion modeling, which gradually improves predictions through self-attention and iterative denoising.
In their evaluation, DIFFUSEARCH was compared to three transformer-based models trained using different methods. The researchers used a dataset of 100,000 chess games, encoding states in FEN format and actions in UCI notation. They implemented a GPT-2-based model with specific parameters and found that DIFFUSEARCH outperformed the others in terms of action accuracy and Elo ratings in an internal tournament.
The results showed that DIFFUSEARCH achieved higher accuracy while using significantly less data than its competitors. The model also demonstrated the ability to predict future moves, although accuracy declined over time. Performance improved with more attention layers and refined decoding strategies. Positioned as an implicit search method, DIFFUSEARCH proved competitive with traditional explicit search methods like MCTS.
In summary, this new framework shows promise in enhancing chess decision-making by replacing explicit search with implicit strategies. While it currently relies on external resources and a limited dataset, there is potential for further improvement through self-play and advanced modeling techniques. This approach could also benefit other areas, such as next-token prediction in language models, opening up new avenues for research in AI planning and decision-making.