Google recently launched an intelligent drawing software called Auto Draw. As the name suggests, the software can draw automatically. Just draw a sketch on the software, and then it will automatically match similar pictures for you to choose. This is a very basic artificial intelligence software, which is very similar to the handwriting input function of a mobile phone.
Drawing cheats?
At the same time, Ali Eslami and several other scientists developed a more intelligent, bionic thinking, and self-learning drawing software. What this software can do is: after providing a picture, it can draw a "copy". The key is the drawing process, that is, the software has the ability to draw images instead of simply scanning and outputting like a scanner.
The following are scientists’ research and development ideas:
inspiration:
We found that humans see more than just the surface of things. For example, when we see a building, we not only admire the complexity of the design, but also appreciate the construction process of the building. This ability to see the essence through the phenomenon is an important reason why human intelligence is higher than other species. Therefore, we hope to make a software that is as smart as humans or imitates human thinking. For example, when looking at a painting, we hope that the software can "understand" how the image is drawn, rather than just distinguishing the pixels of the image.
Preparation: Equipment
We gave the software the same drawing tools and found that it could automatically tell the different ways numbers, people, and portraits are constructed. Crucially, it learned all of this on its own, without being trained on a dataset. This is in stark contrast to other programs that must learn from datasets, which can be time-consuming.
Engineer design ideas
Step 1: Deep reinforcement learning software foundation
We first designed a deep reinforcement learning software to interact with the computer painting program: use an electronic pen to draw on the tablet, and change the brush size, pressure and color at will. However, because it has not been trained, the strokes drawn are all random and the overall structure cannot be seen. To solve this problem, we decided to establish a reward and punishment mechanism to encourage the production of more valuable images.
Step 2: Discriminator neural network upgrade
Then we designed a neural network called a discriminator to determine whether the image is drawn by software or a real photo. If the image drawn by the software is judged to be a real photo, the software will get a "reward". In fact, the "reward" of the software is that it has learned how to deceive the discriminator. This is similar to the method used by the previous neural network GANS (Generative Adversarial Networks), but the difference is that GAN directly outputs pixels, while ours outputs images after the software interacts with the real image environment.
Step 3: Training on MNIST & OMNIGLOT datasets
In the first set of experiments, the software was trained to generate images of digits similar to those in the MNIST dataset. By trying to generate images that fooled the discriminator, the software learned to control the brush to adapt to the style of different digits, which is what we often call visual programming technology.
Step 4: Diffusion to real-person datasets
We also train the software with specific images, asking it to generate "copied" images, and then let the discriminator determine whether the "copied" image is generated by the software and whether it is a copy of the original image. The more difficult the distinction is, the greater the "reward" (the more the software learns). Crucially, the software stores a series of actions to control the simulated brush. This means that what is learned during the simulation can be applied to other similar environments, such as on a robot arm to simulate a real person. The video screenshot is as follows:
Another thing is that the framework can be expanded to real data sets. When training to draw celebrity facial expressions, the software can capture the main features of the face: shape, tone, hairstyle, just like a street artist sketches a portrait with just a few strokes.
Knowing what something is and why it is is an ability that humans easily possess and use frequently. In this study, we successfully enabled the software to "understand" how a painting was drawn after seeing it.During this period, I also learned visual programming techniques and how to express cause and effect relationships concisely.
Although this is just a small study, the subsequent development of such technologies that enable artificial intelligence to have human-like cognitive, generalization, analytical and communication capabilities is essential. Although it is impossible to replace designers in the short term, and even any of the "Adobe family bucket" cannot be replaced, it is undeniable that the emergence of automatic drawing software has indeed shown us another interesting and promising application scenario for artificial intelligence.