How can computer vision systems simplify merchandiser’s work? What machine learning solution will become an alternative to Amazon or Google? We discussed these issues with Sergey Nikolenko, a speaker at AI Conference and Chief Research Officer at Neuromation.

Neuromation is a startup developing a blockchain platform of neural network synthetic data and a system for identification of goods in appearance. In early 2018, Neuromation managed to raise $50 million during the ICO.

Interviewer: AI Conference (AI)
Respondent: Sergey Nikolenko (SN)

AI: Tell us in detail about Neuromation's platform: how and where can it be used?

S.N.: It is a core product of our company. We are developing it to help artificial intelligence specialists. We expect that Neuromation will be a better solution for experts in Deep Learning and other machine learning areas than existing Amazon or Google apps. The platform will allow to lease computing hardware and receive open datasets or buy private ones, as well as produce new synthetic data (we will talk about them later), develop and teach deep neural networks and other machine learning models.

Moreover, we hope that when the platform skyrockets, it will become not just a set of tools but a kind of hub for the community of artificial intelligence specialists. In other words, it will include people able to both use the platform and, for example, accept external orders from companies facing challenges in solving their AI problems. Currently, it is quite difficult to find and hire high-end experts.


AI: The Neuromation startup raised $50 million in just eight hours during the ICO. How can you explain such a success?

S.N.: The main explanation is, surely, our genius sales managers, primarily Evan Katz – our Chief Revenue Officer. The success of token sale is considerably his merit. However, the truth is also that we have had luck with a token sale. It may be said that we have jumped in the last carriage of the leaving train. Shortly after our token sale, all cryptocurrencies and tokens strongly plummeted; the interest in them decreased and no one heard about such success stories. I will not claim that it is the last successful ICO, but it is definitely one of the last ones. In a couple of months after our token sale, the market collapsed.


AI: One of your solutions is a system for identification of goods on the shelf. How does this system help outlet chains?

S.N.: It was our first project. Such an idea has become the core one for internal research of synthetic data used to train neural networks.

The project concept is quite simple. There are millions of merchandisers visiting shops and checking shelves. For instance, they check whether Coca-Cola occupies 30% of its paid space and whether all the bottles face customers. But a person should not deal with such things. This process can be automated using modern computer vision.

When we started the project, the task seemed to be standard: recognition of objects on images. However, we have promptly figured out that there is not just a lack of data for neural network training, but it is impossible to receive them in sufficient quantity: a catalog of goods is enormous, retail lists contain 170 000 items, and it is not possible to collect a set of manually marked images, covering 170 000 different types in various patterns.

This led to the idea of synthetic data. We started making 3D models of goods: for example, a 3D model of a bottle stuck with virtual labels. Therefore, we obtain dozens or more products placed on virtual shelves out of a single model. This is how a training sample of “cartoonish” super market is formed. The know-how is that we manage to teach computer vision models using such “cartoons” and transfer this process into reality.


AI: What investigations related to molecular biology and medicine is Neuromation engaged in? What challenges have you faced during the integration of AI into healthcare?

S.N.:  We have two key focus areas. One of them is computer vision for healthcare, which means processing of various medical images. Some our specialists are considered to be the best in this very sector. We regularly participate in contests and write specialized articles.

Another area is our cooperation with Insilico Medicine. Insilico is a company trying to discover new drugs using modern and recently developed machine learning models. We are already completing a significant collaborative project, and I hope we will be able to introduce it soon.

The integration of artificial intelligence into the healthcare sector is a complicated task. A lot of developers from different areas are complaining about huge cycles of integration in medicine. In fact, it is not referred only to artificial intelligence: putting drugs to use requires a pretty long cycle anyway.

If your neural network designs a drug, you cannot just take it and sell as tablets. It will definitely need laboratory and clinical trials. Fortunately, we do not have to perform them on our own, as there are partners in corresponding industries.

Nevertheless, medicine processes are indeed slow. For instance, Insilico has been long focusing on efficient drug discovery. Over the years, they haven’t had yet any drug passing the full cycle, because the process is incredibly long.


AI: One of Neuromation’s project is satellite image processing. What are the concept and goals of this project?

S.N.: It is a research project. We have unexpectedly, but quite successfully, participated in the DeepGlobe contest held as part of the largest computer vision conference in 2018. It consisted of several tasks related to satellite image processing: segmentation of lands on images as well as segmentation of roads and houses on satellite images. We took high places in these competitions and wrote three articles published in the collection of the corresponding seminar. It was a scientific success of our company.


AI: You have been cooperating with Samsung for a long time. What have you done? What solutions have you been involved in?

S.N.: It rather refers to me than to Neuromation. I have been cooperating with Samsung for a long time. We worked on collaborative projects based on St. Petersburg Department of Steklov Mathematical Institute of Russian Academy of Sciences. I have been working for this institute for 15 years. I am a research scientist (here is the list of my publications). Besides, I constantly give lectures.

Samsung has given grants for artificial intelligence research at Steklov Mathematical Institute for several years running. Moreover, our relations have pushed on to the next level since this year. Now, is supporting the whole lab at the institute led by me.

At the same time, one has recently established the Samsung AI Center in Moscow. It is a brand new division aimed at artificial intelligence. I am leading one of the laboratories in this center remotely and part-time. We are engaged in the analysis of multimodal data, in other words, machine learning in situations where various kinds of data are consequently available: images, texts, sound.


AI: What will you talk about at AI Conference?

S.N.: I will talk about synthetic data: about the primary idea of computer vision promoted by Neuromation. The presentation will be beginner-friendly, without any complex mathematics.

I will also provide examples and explain how the idea is gradually becoming popular. The solution is brand new, curiously enough, although it has always been self-explanatory. And, perhaps, I will reveal how people are trying to produce and improve synthetic data using other machine learning models.


Sergey Nikolenko will speak at international AI Conference that will take place on November 22 in Moscow.

Register