After Looking at 50,000 Album Covers, AI Designer Outputs Crazy

The Spanish artist used StyleGAN2 to create an AI designer, who taught himself with the help of 50,000 images. He didn’t expect that it would be so easy to train an impressionist designer.

Perhaps some designs are so classic and outstanding that people can't help but imitate them.

The day before yesterday, Meng Jia, who had come into the public eye with "Riding the Wind and Waves", released a new song, and the album cover was accused of plagiarizing an Instagram celebrity exactly;
Not long after, actress Song Zuer's styling design was also pointed out to have copied the album cover of a Korean star at pixel level, and even the original watermark was not removed. It can be seen that design creativity is indeed not easy.

People who engage in creative work are bound to run out of inspiration.Intentionally or unintentionally refer to, borrow from, and imitate other people's works. "Can it be considered stealing if it's an artist's work?"
Well, facing the condemnation of netizens, not only did the star lose face, but the designer also lost his reputation.

50,000 album covers, AI designer taught himself

Although both are secondary creations after copying, AI with tool attributes seems to be more justified.

Spanish genius artist Bruno López, based on 50,000 album covers on Spotify have enabled NVIDIA's StyleGAN2 to learn by itself. "Evolved" into an "emotionless, abstract album cover AI designer".

StyleGAN2 quickly formed its own set of design specifications and style genres from 50,000 Spotify album covers.
The real album cover is basically composed of two parts.Text and main visual elements.

The AI training process is similar to how humans “look for patterns in pictures”. In the layout of the album cover,AI also seems to be well versed in size ratio standards such as the golden ratio, Fibonacci numbers, and the rule of thirds.

However, the overall style of the album covers generated by AI is a bit "weird", and some covers can even be described as "horrible": a hideous human face, an eye missing an eyeball, and a background that is unclear whether it is waves or animal corpses...not to mention the blurred text and gloomy tones.

In addition to being affected by the style of the training samples, the appearance of these artifacts is also related to the model itself, because these 50,000 training data are all from Spotify's editorial recommendation list.There is no restriction on the musical art style, which leads to insufficient training data for a single style and more serious artifacts and spots.

StyleGAN is a new generator network structure released by NVIDIA. This model draws on the relevant technology of style transfer and can modify the main attributes of the target object layer by layer.

Due to the existence of instance normalization in AdaIN, some key information of the input will be lost during the style transfer process, which leads to the existence of spots and artifacts in the generated images.

StyleGAN2 is an upgraded version of StyleGAN, which improves the quality of generated images, greatly reduces the appearance of feature artifacts, and is more sophisticated in detail processing.

Use AI to draw the cover and generate a music MV

To train a mature AI designer, training data and pre-trained models are indispensable. Bruno López introduced the birth process of "AI Designer" and made an abstract MV (music video) with these covers.

  Create a dataset:
1.Found the corresponding API in Spotify, which can return multiple metadata, including album cover, given an album ID.
2.Write a crawler script to crawl the album library in Spotify editor's recommendation list and download the album cover.
3.Get a dataset with about 50,000 images and a resolution of 640×640.

Training process:

The author trained the model using the skyflynil implementation of the original StyleGAN2 in Google Colab Pro. The training took several days and stopped after 50,000 epochs.

Simply relying on AI to generate a crude album cover seems to be lacking in quality in the eyes of great artist Bruno López.

The timing video that went viral on social networks gave him a flash of inspiration.

  Make a card point video:

The author wrote a Python script based on lhzbrian to create random latent vectors, and then interpolate these vectors to create an AI-generated animated transition.

In order to synchronize the animation with the song, the author cleverly used PyDub to synchronize the interpolation speed with the beat volume of each frame.

Finally, using Pillow’s Image module and OpenCV’s face detection capabilities, the script can generate images with specified features (such as yellow album cover without faces) using only the latent vector.

Bruno also considered the audience experience very carefully. After all, a clip of a few seconds always leaves you wanting more. In order to make the final film more enjoyable, Bruno thought of permutations and combinations.

He first generated multiple clips with different attributes, and then compressed them into a video using Premiere.This is the final "Interference Music Video" film.

Style migrates, creativity also migrates?

In art, where is the boundary between creation and imitation?

From the creative process, artistic creation starts with copying, copying nature, copying society. In the process of learning to generate album covers, StyleGAN2 fully reproduces the process from copying others to forming a personal style.

The painter Qi Baishi advocated that the beauty of art lies in "the balance between similarity and dissimilarity". His painting style is fresh, vivid and unique. In fact, Qi also started by copying other people's paintings in his early years.Through constant appreciation, copying and thinking, I gradually found my own personal style.

On the other hand, the "design porters" plagiarize pixel by pixel, put their own names on the designs, and even use them for commercial profit. In terms of business ability and professional ethics, these creators are indeed not as good as AI.

refer to:https://arxiv.org/pdf/1912.04958.pdfhttps://thenextweb.com/neural/2021/03/08/ai-music-video-deep-learning-algorithm-stylegan2-album-covers-spotify-gans/https://docs.google.com/document/d/1qePvtKTJ3lbpxKfiJ1cn0wAVLazza0Zux4Lb-aTHuBE/edit?usp=sharing