Co-founder of OpenAI nonprofit project Greg Brockman believes that computer game self-learning bots demonstrate huge capabilities of artificial intelligence in an efficient and safe way. So how e-sports is developing machine learning?

Machine learning allows computers to work without direct programming. In fact, it’s the development of artificial intelligence that does not rely on preset schemes but analyzes data and takes relevant decisions on its own. Machine learning algorithm uses different approaches that show different efficiency.

Training of artificial intelligence is also influenced by conditions, in which the process takes place. The most interesting and difficult task for researchers is machine learning in dynamically changing conditions. Most of modern artificial intelligence models act within strict boundaries, and their training requires the use of huge data arrays, the generation of which takes a lot of time.

For this reason, virtual reality is convenient to use as a platform to train artificial intelligence. Specifically, the popular game Dota 2 is used as a base for machine learning.

Machine learning in Dota 2: advantages of the platform

OpenAI nonprofit organization builds serious bots for Dota 2. The project researches capabilities of artificial intelligence and provides an open access to it. The research group, with Elon Musk as one of the founders, focuses on making artificial intelligence exclusively useful.

Founded in 2015, the project has been engaged in the work with artificial intelligence by studying, developing, and advancing the technology (in particular, for robotics). Dota 2, for which OpenAI is developing bots, gives AI the maximum of opportunities to realize its potential. The platform offers the following advantages for machine learning:

  • e-sports is popular and many people generate data than can be used;
  • such data (examples of real people playing, used to train bots) is freely available;
  • the platform is fully secure for testing and learning, interaction of a person with AI;
  • the game mechanics of Dota 2 is very diversified and gives a possibility to drill different scenarios and methods for training of AI.

The key item on this list is the availability of a huge, constantly expanding database of examples generated by real players. The efficiency of machine learning directly depends on the samplings that it is based on.

AI Conference: Machine learning in e-sports: reasons for use 1

Artificial intelligence vs Dota 2 players

The approach used by OpenAI in this project is based on neuroevolution. In this form, machine learning applies an evolutionary algorithm for training of artificial neural networks. It uses the following principle: by making random actions, the network forms a number of various decisions, which in their turn are assessed by the fitness function. In other words, developers built a bot, implemented it in Dota 2 and gave it an opportunity to take decisions on its own.

In e-sports, such bots can successfully compete with people, which was proven by the results of Dota 2 tournaments. For example, OpenAI Five gained the victory scoring 2:0 in the play against e-sports players in San Francisco. In the official blog, representatives of OpenAI explain and show how they managed to level up the AI-based bot in two weeks of intensive practice and teach it to play better than professionals.

The project’s CTO Greg Brockman believes it is a serious step towards the development of complex AI capable to solve important tasks in different areas, for example, in healthcare.

Machine learning and poker

Another example of machine learning use are poker bots. This game is characterized by incompleteness of data about cards and participants and involves the element of chance. Poker requires nontrivial, multilevel approaches to problem solving from the artificial brain, so that is why many developers are conducting research in this field.

For example, representatives of the University of Alberta developed DeepStack algorithm that can be applied for no-limit heads-up poker. It works basing on deep neural networks, can learn, among other things, during games with itself. This algorithm has shown good results in practice by beating professional poker players and advancing. Carnegie Mellon University built Libratus system based on the algorithm that managed to beat some of the world’s best players of no-limit heads-up poker.

These solutions and developments are important for not only e-sports and online games, but for the development of artificial intelligence in general. Algorithms of poker bots based on machine learning can be also applied in marketing, financial management, security. In short, they can be used in any area that requires teaching agents in the environment with incomplete information, for example, in risk management apps. In this case, poker algorithms will help artificial intelligence to take optimal decisions in uncertain conditions.

Conclusions

Machine learning is most efficient in dynamically changing conditions. The approach most commonly used for machine learning in e-sports involves neural networks and genetic algorithm. It is based on the creation of a “generation of decisions” based on a successful choice. Algorithms developed basing on this principle can be used in not only virtual reality but also serve as a useful tool in various areas of life.