Hi everyone! Just thought i'd do a writeup for a hackathon project I recently worked on.
The hackathon was a mini hackathon (half day) so we didn't really have that much time to actually develop something. I've done quite a lot of work on discord chatbots and recently have been working quite a lot with OpenAI so we decided to see how well GPT-3 performed at giving a user a quick sanity check of emails to tell them if they are likely to be a scam, and if so how the scam worked.
I already have a chat bot I use for rapid prototyping of ideas hooked up to OpenAI text-davinci-003. We weren't really expected to actually build anything but they didn't have anything against us using existing code so we decided we'd give it a shot anyway.
OpenAI is really meant for text completion. For every message a user sends, it uses the following format:
.prompt
promptmessage = ".prompt You are an AI chatbot designed to scan incoming messages and assess if they are likely to be spam or a scam. Your job is to alert the user of the probability, as well as provide an explanation of how the scam works. You should give advice on how to proceed."
content = f"{promptmessage} {content}"
If you are an admin, you can set the promptmessage by using the command .prompt. This replaced the promptmessage which is really useful for rapid prototyping without having to restart the program.
For initial testing, I went to my spam box and grabbed text from a bunch of obvious scam emails as well as trying a few of the classic well known email scams. Here are some examples.
As you can see, with scam emails it generally performs pretty well. I also tried a few real non-spam emails to give it some negative tests.
Great! So it also seems to perform pretty well in negative tests. With the given prompt, it also gives general advice on how to proceed even if it is not a scam.
We didn't really have that much time to flesh out the idea, but we were the only team to actually have a project to demo and won the hackathon!
Obviously it is pretty limited in scope, but I definitely feel like GPT-3 has great potential in detecting (as well as generating) spam emails. I can see the possibility of a plugin for outlook / gmail that adds a little header to the top of every email giving a TLDR description as well as advice on if / how to respond.
Top comments (0)