Can AI Poet Write About the Mood of Mid-Autumn Festival?

Perhaps everyone has a poetic dream in their heart, hoping that they can express their feelings with refined verses like the ancient literati. For example, during the Mid-Autumn Festival, write a poem to send blessings to relatives and friends. But unfortunately, writing poetry is not as simple as imagined. However, artificial intelligence technology can already fulfill your poetic dream. Huawei Noah's Ark Laboratory's latest "Yuefu AI" can generate a poem in a few seconds as long as the theme is given. Come and write a poem!

How do you spend the Mid-Autumn Festival? This festival of reunion and moon-viewing has inspired the feelings of countless literati from ancient times to the present, leaving behind many meaningful poems.

Facing the beautiful scenery of the Mid-Autumn Festival with its full moon and beautiful flowers, the ancients were able to chant "The bright moon rises over the sea, and people all over the world share this moment"The most beautiful poem, but now most people can only helplessly turn their infinite feelings into a sentence "Ah, the moon is so round."

This is probably the distance between "The setting sun and the lone wild goose fly together, the autumn water and the sky are the same color" and "What a big bird."

But don’t be sad, now AI can help us bridge this distance.letEveryone can easily compose poems and become a poet!

Huawei launches the "Yuefu Poetry" applet

recently,Huawei EI Experience Space applet has launched a new feature - Yuefu poetry.As long as you give a theme, select the number of words and the genre (quatrain or regulated verse), you can create a poem in an instant. Mid-Autumn Festival:

Moon Viewing:

Machine Learning:

"AI":

If I don't tell you, can you tell that these poems are written by AI? In June this year,Huawei Noah's Ark LaboratoryPublished a paper"GPT-based Generation for Classical Chinese Poetry", which details the training process of the model and the presentation of the results.

In general, the key technology behind this poetry generation model lies in the powerful GPT model.

GPT is composed of OpenAI  The core concept of the proposed pre-trained natural language model is to first use unlabeled text to train the language model, and then fine-tune the model through labeled data according to specific tasks.The team first used a large-scale Chinese news corpus to train a Chinese GPT model.Then we used Chinese classical poetry as a corpus and fine-tuned it to get the poetry generation system - Yuefu AI.

Moreover, in order to enable AI to learn to write poetry on its own, this GPT model does not have any settings. The number of words, rhythm, prosody, and antithesis knowledge required for writing poetry are not set. The model learns all of these from ancient poetry data on its own.

Compared with "Yuefu AI", humans who have studied Chinese for many years but still cannot write a decent poem really have to feel inferior. 

A detailed account of the process of AI learning to write poetry

The generation of classical Chinese poetry is an interesting challenge in the field of natural language generation.

Different from free text generation, Chinese classical poetry usually has to meet certain requirements in form and content.Number of words, rhyme, tone and parallelismEtc. In addition, the content also requires that the theme of the entire poem is consistent and coherent.

In addition, there are many forms of Chinese classical poetry, including five-character quatrains, regulated verses, seven-character quatrains, regulated verses, and ci tunes such as Man Jiang Hong and Shui Tiao Ge Tou, as well as couplets.It is relatively complicated to use artificial intelligence methods to produce classical Chinese poetry.Scientists from Huawei's Noah's Ark Lab studied the problem of poetry generation with given forms and specific themes, and proposed a poetry generation method based on the pre-trained model GPT.

The specific details of the method are:"Yuefu AI" uses Transfomer as the basic model and implements its own GPT model based on the BERT source code.The Transformer size is the same as BERT-Base, and the tokenized script and Chinese vocabulary released in BERT are also adopted. For text generation, they implemented truncated Top-k samplingto generate different texts.

Learning to write poetry with "Yuefu AI" is divided into the following three stages:

Phase 1: Data Processing

Huawei Noah's Ark Lab's GPT model is trained with Chinese news corpus. The data training is mainly divided intoPre-training and fine-tuningTwo stages. For fine-tuning, they collected publicly available classical Chinese poems and then converted the sample poems into formatted sequences.

Phase 2: Model Training

Next, they pre-trained their GPT model on a cloud service with a news corpus, which took 90 hours. They then trained it for 4 epochs using 8 NVIDIA V100s (16 GB). After training, they fine-tuned it by feeding all the poetry training sequences into the transformer and training an autoregressive language model to observe that the probability of any sequence X is maximized:

Stage 3: Poetry Generation

Once trained, the model was applied to generate poems on a specific topic upon request. They first converted the form and topic into [form, identifier1, subject, identifier2]The initial sequence is then given to the model and the body fields are decoded one by one.

During decoding, no hard constraints are required to ensure formal correctness. Instead, the model is able toAutomatically assign positions for commas and periods.When the poem is finished, it will be 「EOS」The token ends the process.

"Yuefu AI" not only learned how to compose poetry, but also mastered a new skill - acrostic poetry.

For the acrostic poems, they still used the same method to train them. However, when training the acrostic poem model, they replaced the themes in the previous training process (such as "静夜思") with the first letter of each sentence of the poem (such as "床疑举低"). 

Surprises brought by AI poets

The team was surprised by the results generated by this approach.

They found that the model had learned the complex pairing rules of regulated verse, which is difficult to master even for people whose native language is Chinese and who have received normal education (I believe many students feel the same way).

In the generated quatrains and regulated verses, 95% and above performed well, but were slightly inferior in terms of ci.The reason may be that, compared with quatrains and regulated verses, the form of cipai is relatively more complex, and the training samples are far less than the former.

There are tens of thousands of training samples for both quatrains and regulated verses.There are only 882 types of ci tunes.Among them, only 104 species have more than 100 training samples, and even the largest sample contains only 816.

In addition, within a given topic, the generated results are usually very good, but for some topics, such as "machine translation", it is difficult to generate poems because they rarely appear in the training corpus. On the contrary, topics such as "Autumn Thoughts" are easy to generate.

The team introduced,This method is much simpler than the existing method based on recurrent neural network (RNN).And can produce better poetry.

AI learning to write poetry has long existed

Some people say that China is a country of poetry. From ancient times to the present, scholars and young people have always been passionate about writing poetry. And the majority of science and engineering students have also been constantly using technology to express their romantic feelings.

In December 2017, a poetry robot developed by a team from Tsinghua University "Nine Songs"He appeared on the CCTV program "Super Smart" and his poems brought great surprise to the audience.

"Jiuge" is based on deep learning, and learned to write poetry after learning more than 800,000 poems. It has the characteristics of multi-modal input, multi-genre and multi-style, and human-computer interactive creation mode.

In addition to ancient poetry, modern poetry also seems to be no problem for AI, such asMicrosoft XiaoIce has published a collection of poems.There are also AI poetry generation tools in multiple languages abroad. 

Can AI really write poetry, or is it just making it up?

AI writes poetry, some people praise it, while others criticize it.

Liu Luning, a director of the Shanghai Poetry Society, once said that humans cannot compete with computers in terms of speed when it comes to writing poetry, but a poem he wrote in one day cannot be matched by a computer writing a thousand pages. Director Deng Wanying also said that the characteristic of AI poetry is that it has poems but no text.

Compared to the works of men of letters and poets when their inspiration is flowing, the poems written by AI can be said to be "writers without emotions". They discover the patterns of poetry in a large amount of text and "piece together" complete but potentially meaningless verses.

But for researchers, AI poetry writing is more of a display of technical capabilities, and they value the development of NLP technology behind AI poetry writing.

In addition, for ordinary people, AI poetry writing tools also allow everyone to create and express their feelings through poetry, which is also a good tool for entertainment. Let us also give AI some time, and believe that it will bring more surprises to the world.

-- over--