As reported by VentureBeat, machine learning is on its way to changing our perception of gaming possibilities. Vladimir Ivanov, Sr. Deep Learning Engineer at NVIDIA shares this opinion. He works in a QA department helping 80 m GeForce Experience users play video games with optimal settings. To attain this result, he applies reinforcement learning of neural networks.

In his interview to the AI Conference press service, Vladimir Ivanov shared his memories of the first courses on development and told about the experience with neural networks at NVIDIA. He also revealed what AI developers actually spend 90% of their time on.

“In my opinion, regression and neural networks solve the same tasks: we want to make something clear and search for solutions to these tasks. Only tools are changing”

I graduated from the Faculty of Physics of the Lomonosov Moscow State University where I worked a lot with data. It used to be called much simpler – statistics and regression.

As soon as more complex tasks aroused, computational capacity grew. Graphics cards were introduced for data processing, which somehow contributed to the appearance of neural networks. New functions of them became available allowing to solve more tasks. We used to analyze hundreds and thousands of points, and right now millions of them can be investigated. Such a smooth shift form regression and statistics resulted in the AI appearance.

In my opinion, regression and neural networks solve the same tasks: we want to make something clear and search for solutions to these tasks. Only tools are changing.

“Beginning developers should undergo courses with homework”

Beginning developers should undergo courses with homework. For example, Stanford University currently offers plenty of courses with publicly available video lectures and home tasks presented as Python scripts. I recommend CS231n course with a clear explanation of neural networks. That would help if you want to understand the operational principle of a certain system.

Of course, such courses didn’t exist ten years ago. Nonetheless, I managed to learn from Stanford-based lectures called ‘Machine learning intro’ prepared by the American information scientist Andrew Ng.

“AI helps us extract information from a visual sequence – for instance, whether it is comfortable for people to play with certain settings or not”

I joined NVIDIA two years ago. At the company, I work with machine learning in a QA department. New technologies are constantly emerging here. Our task is to find out improvements in a new version of graphics cards and drivers.

What is more, NVIDIA has a GeForce Experience recommendation system helping figure out the best settings for video games. Working on this project, I apply computer vision for their analysis.

Roughly speaking, AI helps us extract information from a visual sequence – for instance, whether it is comfortable for people to play with certain settings or not.

The operational scheme is as follows: we have a game and take out a certain image (screenshot). It depicts actions of some characters. A neural network recognizes the number of objects, which would help detect a level of event saturation of a scene. The more objects are in a scene, the more loaded a graphics card is.

“I can talk about reinforcement machine learning for hours”

When we test graphics cards, we want to see what experience a gamer would receive. Therefore, to automate this process, we need to train a program to simulate human behavior.

That is why we train neural networks.

There exist various kinds of machine learning – for example, supervised training. In brief, we set certain parameters for AI: ‘move right’, ‘look at the picture’, etc. Thus, a program performs the specified actions but cannot set them independently.

Imitation of human behavior is almost an impossible mission for us. We cannot teach an algorithm ‘in such situations a man behaves in this way’ because it’s difficult to predict players’ actions in certain cases.

However, we can single out human behavioral archetypes. Consequently, we may invite an experienced gamer and let a neural network monitor him or her.

I can talk about reinforcement machine learning for hours. A neural network trained by means of this method is rewarded for performed actions.

Let me exemplify it on a race: its main idea lies in driving as fast as you can. Our neural network enters a game and gets, say, 1 ruble for each meter. In shoot-em-ups, the number of target hits affects the score.

We reward a network for actions that win points. We do not tell it ‘in this situation, turn to the left’. It independently detects what action to perform in order to gain points – just as a real player would do.

If an untrained neural network enters a game – be ready for surprises.

“Postgraduates and students spend 90% of time on models development while we spend 90% of time on data processing”

Data quality is vital, especially for reinforcement learning. Postgraduates and students spend 90% of time on models development while we spend 90% of time on data processing

Virtualization is a formula for data success. Images should be sorted, textual data, say, speed measurement and the number of cadres per second in a video game should be visualized via infographics.

When undergoing one of the courses, I decided to publish several home tasks on GitHub, which attracted talent hunters. That is how I got engaged in neural networks.

Vladimir Ivanov will deliver a presentation at AI Conference. He will cover the difference between RL and Supervised and Unsupervised Learning, the nature of Imitation Learning and its application at NVIDIA and Tesla. Video examples will illustrate all the theory.


To meet Vladimir Ivanov and other industry experts, register to AI Conference scheduled for November 22 in Moscow.

Register