Publication: Jugadores automáticos basados en árboles de búsqueda de Monte Carlo
Loading...
Official URL
Full text at PDC
Publication Date
2018-09
Authors
Advisors (or tutors)
Editors
Journal Title
Journal ISSN
Volume Title
Publisher
Citations
Exportar
Abstract
Dada la enorme cantidad de juegos existentes, una pregunta recurrente ha sido ¿Cuál es la mejor estrategia a seguir para maximizar mi beneficio? en otras palabras, ¿Qué acciones debo elegir entre las posibilidades que tengo para obtener la mayor recompensa posible? Esta cuestión, al darle una perspectiva formal resulta no ser trivial, cada configuración del tablero tiene una probabilidad asociada de ganar, definida por todos los posibles desenlaces que tiene el juego desde esa posición. Sin embargo, obtener este valor es imposible computacionalmente ya que esto implicaría revisar todas las posibles partidas dado que el número de estas es descomunal. Para dar solución a esto, existen diversos algoritmos que intentan aproximar este valor, normalmente mezclando el conocimiento previo del dominio con la búsqueda exhaustiva en el conjunto de todas las partidas, para comprobar si una acción mejora nuestra situación actual.
En este trabajo se presenta Monte Carlo Tree Search, un algoritmo que introduce una forma completamente distinta de estimar la probabilidad de ganar de cada estado de la partida. El método será generar simulaciones de partidas en las que las decisiones se toman de forma aleatoria. De esta forma, podremos calcular el porcentaje de veces que una simulación resulta victoriosa en comparación con las veces que se ha simulado, dándonos así una aproximación del valor buscado. Veremos que cuantas más simulaciones lancemos, el resultado será más ajustado a la realidad. Conoceremos 2 versiones, la versión dinámica o MCTS, y la versión plana o PMCTS. Introduciremos métodos de optimización y una perspectiva teórica del funcionamiento.
Probaremos este algoritmo en juegos, y para ello obtendremos diversas implementaciones de las 2 versiones y las adaptaremos para jugar al Reversi y al Ajedrez. Los adversarios a batir serán algoritmos varios que irán aumentando gradualmente su dificultad y nos obligarán a ir adaptando los algoritmos y mejorándolos para que sean capaces de dominar.
Por último introduciremos algunos algoritmos utilizados en el campo de la Inteligencia Artificial conocido como aprendizaje automático o Machine Learning, para que el lector posea las herramientas necesarias para entender el funcionamiento del jugador automático más potente que existe en la actualidad: AlphaGo. Este programa ha sido capaz de vencer a los mejores jugadores del mundo en el Go, considerado hasta ahora como imposible de abarcar para la inteligencia artificial. Monte Carlo Tree Search es uno de los elementos clave en este código y veremos cuál es su papel.
Since the beginning of the last century, classic games were studied from a formal perspective adding mathematical reasoning. With the arrival of computer sciences and artificial intelligence, creating automatic players has been a recurrent problem. The part of mathematics which studies this kind of situations is called Game theory and it include classifications, representations and solutions to how to optimize the decision making. One of the typical representations in games where the players take turns to make moves is using a tree, the way to explore it is the base of how to obtain efficient players. Some classic exploration algorithms may work for simple games, but for those with more than one player, we will need approaches like Minimax. In this work we will present Monte Carlo Tree Search, an algorithm based in avoiding to explore the whole tree because this is impossible and, instead it will try, using random game simulations, to approximate a numeric value of how useful each possible move really is. We will discover 2 versions, the dynamic version or MCTS, and the plain version or PMCTS. We will introduce methods of optimization as well as a theoretical perspective of why it works. We will try the algorithm in game, for that we will obtain several implementations of both versions and we will adapt them to play Reversi and Chess. The opponents to beat will be diverse algorithms that will gradually increase their difficulty and will make us adapt our tools and improve them so we are able to win. Finally, we will introduce some algorithms used in the Artificial Intelligence field known as Machine Learning, so that the reader owns the tools to understand how the most advanced player nowadays works: AlphaGo. This program has been able to beat the top Go players, considered until now as impossible to encompass by artificial intelligence. Monte Carlo Tree Search is one of the key components in this code and we will explain it’s paper.
Since the beginning of the last century, classic games were studied from a formal perspective adding mathematical reasoning. With the arrival of computer sciences and artificial intelligence, creating automatic players has been a recurrent problem. The part of mathematics which studies this kind of situations is called Game theory and it include classifications, representations and solutions to how to optimize the decision making. One of the typical representations in games where the players take turns to make moves is using a tree, the way to explore it is the base of how to obtain efficient players. Some classic exploration algorithms may work for simple games, but for those with more than one player, we will need approaches like Minimax. In this work we will present Monte Carlo Tree Search, an algorithm based in avoiding to explore the whole tree because this is impossible and, instead it will try, using random game simulations, to approximate a numeric value of how useful each possible move really is. We will discover 2 versions, the dynamic version or MCTS, and the plain version or PMCTS. We will introduce methods of optimization as well as a theoretical perspective of why it works. We will try the algorithm in game, for that we will obtain several implementations of both versions and we will adapt them to play Reversi and Chess. The opponents to beat will be diverse algorithms that will gradually increase their difficulty and will make us adapt our tools and improve them so we are able to win. Finally, we will introduce some algorithms used in the Artificial Intelligence field known as Machine Learning, so that the reader owns the tools to understand how the most advanced player nowadays works: AlphaGo. This program has been able to beat the top Go players, considered until now as impossible to encompass by artificial intelligence. Monte Carlo Tree Search is one of the key components in this code and we will explain it’s paper.
Description
Universidad Complutense, Facultad de Informática. Departamento de Ingeniería del Software e Inteligencia Artificial, curso 2017/2018