ClippyAI - Developing a Local AI Agent

Introduction

As a developer, I’ve always been passionate about creating tools that solve real-world problems. But there was one issue that consistently irked me: the never-ending stream of repetitive emails. Whether it was customer inquiries, tech support requests, or project updates, my inbox overflowed with similar questions day in and day out. It was like Groundhog Day, but with email threads.

The Annoyance Factor

Picture this: You’re sipping your morning coffee, already diving into some exciting coding challenges, and suddenly, ping! —another email lands in your inbox. It’s the same query you’ve answered a hundred times before. You sigh, type in your well-crafted response, and hit send. Rinse and repeat. It’s not just time-consuming; it’s soul-draining, because you totally lose focus of your coding work.

The Eureka Moment

One fateful afternoon, as I stared at my screen, contemplating the meaning of life (and another email), it hit me: Why not build an AI agent that assists you on these repetitive tasks?
Of course could I just use ChatGPT, but that's notowed at my company due to data privacy reasons. It's also very distracting and time-consuming to navigate to the website and copy and paste the source email, ask ChatGPT to write an answer and copy and paste the reply back to your email application.

The data protection part is easily solvable: With today's modern CPUs and GPUs it is possible to use Ollama and host the interference of an AI model of your choice locally at reasonable speed.

But how to integrate an agent for Ollama into your OS?

The DeepL Windows desktop app came into mind, where you just hit Ctrl+C twice and the app instantly translates the text you selected.

So my idea was to create a daemon who watches the clipboard for changes and then sends the content along with a task description to Ollama.

ClippyAI wouldn’t just suggest — I wanted it to take action. When I hit reply, it would automatically type out the response. Imagine the joy of watching ClippyAI do the grunt work while I sipped my coffee.

I chose the name ClippyAI as a mixture of "Clipboard" and "AI" and it was also inspired by the nostalgic Microsoft Office paperclip, which everyone hated these days.

Because I'm mostly a .NET developer, I used .NET 8 as foundation. I wanted to create a multi-platform application, because I'm using Windows at work and Linux at home, so I chose the Avalonia framework for this project.

After I realized, that the main idea was working well, I extended the ClippyAI's tasks to not just answering emails, but to also explain or translate the copied text or even to do some custom user defined tasks with it.

Key Features

• Clipboard Integration: ClippyAI monitors your clipboard activity in real-time. Whenever you copy text, URLs, or other content, it automatically sends it to the Ollama AI model for analysis.

• Context-Aware Responses: Modern AI models such as Llama3 or Gemma2 are able to consider the context of your task, if you give them enough input. Whether you’re drafting an email, writing code, or composing a document, ClippyAI can provide relevant and accurate responses.

• Workflow Enhancement: By automating repetitive typing tasks, ClippyAI frees up your time and mental energy. Say goodbye to monotonous copy-paste routines!

Getting Started

• Install Ollama from https://ollama.com.
• Run ollama pull gemma2 on the command line to install the gemma2 AI model on your PC.

• Clone the ClippyAI repository from https://github.com/MrDoe/ClippyAI and build and run it in Visual Studio or VS Code. If you are using VS Code, you need to build it twice, because during the first build the necessary resources files will be generated.

• Check the configuration in App.config. The standard values should work fine for most installations.

Early Development Phase

While ClippyAI shows some promise, it’s essential to note that it’s still in its early development phase. As with any cutting-edge technology, there are risks involved. Here’s what you should be aware of:

• Use it at your own risk: ClippyAI is experimental. It may occasionally produce unexpected results or errors. Always double-check the generated content before finalizing it.

• Document safety: ClippyAI may unintentionally delete or overwrite your existing documents, if you are using the keyboard output and Auto-Mode. So be careful where to place your cursor!

• Known issues: German umlauts and other special characters are currently not typed in keyboard mode under Linux/X11.

• Developers Wanted: ClippyAI is an open-source project that is open for contributions from developers like you. If you want to join the development, clone the repo and submit a pull request!

Conclusion

ClippyAI is still a work in progress. It won’t win any Turing Awards yet, but it’s a little side project I want to extend further. So, the next time you receive a prompt reply from me, know that ClippyAI is doing its thing. And if it ever goes rogue, blame the coffee.

Disclaimer: ClippyAI may occasionally channel its inner HAL 9000. Use at your own risk.