[GPT-2] Fine-tuning a model with President Trump's Twitter and making a fake Trump bot with GPT-2 , which is "too dangerous"

Background

By using GPT-2, which is published by OpenAI, We can generate very natural passages.

According to Cornell University survey, some say that 70% of people who read the text generated by GPT-2 misunderstand the text as an article in the New York Times.

At first, full size model was not published because developers guessed that it would have risk used with bad purpose.
However, fortunately, the full size model (the largest 1.5 billion class) model has been already released and availeble.

However, in fact, we are not sure about how this AI can be used effectively.
Therefore, I developed and released a web application called "Mockers", an online tool that anyone can easily use GPT-2.
This will provide an opportunity to consider how to use GPT-2.

First of all, if you want to see what GPT-2 is, try this Mockers generation tool.
https://mockers.io/generator

Please refer to below links to understand how to use.
https://doc.mockers.io/archives/1966/
https://doc.mockers.io/archives/1987/

Purpose

Share the results of experiments using Mockers for fine-tuning.

Fine-tuning is to provide additional data using a model that has already been learned, perform learning at low cost, and generate another model.
A model is created that learns the context and style of the given sentence and generates the sentence in a way that follows.
Mockers does not just try GPT-2, but also supports fine-tuning and automatic posting.

Usecase

By using this mechanism, for example, the following use cases are realized.

-You can build a media that receives Page Views outbreaks so that it does not infringe on the copyright of a curated media, imitate it, and parasitize it.

-You can build a bot that keeps impersonating a certain Twitter account.

Experiment

In this article, as a demonstration, to experiment with fine-tuning using GPT-2,
I used Mockers to finetunin about President Trump's Twitter and create a President Fake Trump bot.

Here, too, you can always see the latest President Trump's Mock.
https://mockers.io/timeline

Procedure

Please visit here.
https://mockers.io/login

You need to login for fine-tuning. Sign up or use a Google account.

If the login is successful, you will be prompted to create a model, so press “Create screen”.

When the new model dialog is displayed, input "Model name" as appropriate and set "Model type" to "Custom model (Twitter)". This allows you to generate a fine-tuning model for your Twitter account. On "Model Settings", In "Sync Target Account (input)", enter the target Twitter account.

Currently, it takes up to two hours to generate a model.
When a model is generated, it is automatically generated periodically, but you can also register an account to tweet the generated text.
To do this, you need to register with Twitter API in advance.
See the following article for how to apply.
https://dev.to/twitterdev/using-the-twitter-api-to-make-your-commute-easier-3od0

The Twitter account entered in "Sync Target Account" triggers original your account to generate a sentence that is related to the tweet to Twitter.

In this way, "mock" is realized.

Result

Here's what Fake Trump actually tweeted: It's not necessarily what he says, but sometimes he says something appropriate to his position, and otherwise it's generated with topics related to what he said in the past You can see that.

Problem

• GPT-2 can control the length of each word, but cannot control the number of characters, so it cannot be optimized with media that is severe in character length such as Twitter. As a result, if you exceed the Twitter limit of 280 characters, you have to force cut it.

• The GPU memory required for fine-tuning is too large, and models larger than 774M will not work with GPUs that can be used by ordinary people. Even the personally owned "Geforce GTX1080 Ti" and AWS P3 instance "Tesla V100" did not work due to lack of memory. (Normal prediction can work well)

In the end

It is expected that sentence generation technology using large-scale unsupervised learning will continue to improve its accuracy and shift to the phase of practical use in the future. We hope this article and Mockers can contribute to natural language AI, development and social implementation at all.

P.S.
Don't forget Hillary.

https://mockers.io