Policy or Value ? Loss Function and Playing Strength in AlphaZero
Por um escritor misterioso
Last updated 25 setembro 2024
Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.
AlphaZero, a novel Reinforcement Learning Algorithm, in JavaScript, by Carlos Aguayo
AlphaZero Explained · On AI
Reimagining Chess with AlphaZero, February 2022
Value targets in off-policy AlphaZero: a new greedy backup
Frontiers AlphaZe∗∗: AlphaZero-like baselines for imperfect information games are surprisingly strong
AlphaGo Zero – How and Why it Works – Tim Wheeler
Frontiers AlphaZe∗∗: AlphaZero-like baselines for imperfect information games are surprisingly strong
AlphaGo Zero – How and Why it Works – Tim Wheeler
Reimagining Chess with AlphaZero, February 2022
Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self- play
Does the neural net of AlphaZero only evaluate the score of a given chess position or does it do something else? - Quora
Recomendado para você
-
AlphaZero, Vladimir Kramnik and reinventing chess25 setembro 2024
-
AlphaZero really is that good25 setembro 2024
-
Inside the (deep) mind of AlphaZero25 setembro 2024
-
Has the Alpha Zero chess program been made to play the Evans Gambit against itself, in an attempt to discover whether that gambit, with best play, is theoretically sound or whether White25 setembro 2024
-
Free Course: DeepMind's AlphaGo Zero and AlphaZero, RL paper explained from Aleksa Gordić - The AI Epiphany25 setembro 2024
-
Contributing to Leela Chess Zero. Creating the Caissa of Chess engines. - Leela Chess Zero25 setembro 2024
-
AlphaZero: Shedding new light on chess, shogi, and Go - Google25 setembro 2024
-
Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self-play25 setembro 2024
-
engines - Alpha Zero vs Lc0 - time for self-play - Chess Stack Exchange25 setembro 2024
-
Why Artificial Intelligence Like AlphaZero Has Trouble With the25 setembro 2024
você pode gostar
-
Halo Infinite release date circles fall 2021, new look at multiplayer & weapons25 setembro 2024
-
Ator Ray Stevenson, da franquia Thor, morre aos 58 anos, Entretenimento25 setembro 2024
-
Bear with me! - 9GAG25 setembro 2024
-
Me Chame Pra Dançar Me Leva Pra Namorar Sacode Meu Corpo Saliente.25 setembro 2024
-
SONIC SUPERSTARS, PC Steam Game25 setembro 2024
-
MC CABELINHO - MINHA PRETA DA DE DEZ A ZERO NAS PATY DE OLHO AZUL25 setembro 2024
-
eFootball 2024 for Android - Download25 setembro 2024
-
Modena-Cagliari 2-0: la decisione del Giudice Sportivo sul ricorso25 setembro 2024
-
El blog de ajedrez de Luisón25 setembro 2024
-
16,523,000+ Background Pictures25 setembro 2024