Google taught the car to draw children's drawings. This is how it works

The participant of the Google Brain team David Ha published a story about how he and his colleagues managed to teach artificial intelligence to draw primitive drawings - they get to children when they pick up the felt -tip pen. Medusa briefly talks about the work of researchers and its possible areas of application.

How does this work

To teach artificial intelligence to perform some kind of operation, you need to give it as many examples as possible. If the data is enough, the program will find patterns and adjust its algorithms so as to give out the desired result.

In the case of the Sketch-RNN program, which David Ha describes, primitive drawings made by hand-in particular, cats and pigs were used as incoming data. The neural network was taught to convert the incoming drawing into a vector consisting of a number of numbers - and then recreating the original image from this vector.

Then the researchers began to introduce distortions to the incoming vector - this taught the program not to “blindly” copy a person’s drawing, but to try to create their drawing, while retaining the properties of the original. In this figure, you can see how the computer painted cats after the person:

Having seen enough of cats, the program learned to understand them well. When she was asked to repeat something absurd, like an animal’s head with three eyes, Sketch-Rn painted a similar pattern, but with two eyes. When the car was shown a toothbrush, she tried to turn it into a cat - but to keep the features of the brush.

The same thing worked in the drawings of pigs: the researchers failed to conduct an algorithm with either a ten -legged animal or a truck (in the latter case, a pig that looked like a truck turned out).

Since the work is carried out with vectors, unusual experiments can be put on incoming drawings. For example, add up and subtract parts of animals!

Researchers took two vectors - one for the head of the cat, and the other for the body of the pig - and gave the program intermediate values. In response, she portrayed a transformation: the cat has grown a body, tail, paws and a patch.

Why is it necessary

Since the algorithm reacts to every obstacle in the incoming vector with a new pattern, artists will be able to very quickly create sets of similar, but unique sketches. It is enough to give the program an example.

You can also use the fact that the neural network, which was taught only in the drawings of cats or pigs, will see cats or pigs everywhere. If you draw a few chairs and ask for the program to repeat the program, it will try to portray cats of the same shape-the same thing will happen as with a carpal pig.

You can also make Sketch-Rnn draw drawings for a person. The image below red shows what people painted, and the rest of the flowers - a painted program. Moreover, it can turn the same sketches into different objects. And as many times as you like.

Finally, the important property of the drawings obtained by the algorithm is associated with the format of incoming data. The fact is that she was handed over not to sets of pixels that form a particular drawing. She was transmitted information about how to move the pen along the canvas in order to get each image. Therefore, in response, it gives out the same information: vector illustrations from Sketch-RNN can be used, for example, to teach people to draw.

Sultan Suleimanov