DEV Community

Cover image for Evaluating LLMs and Prompts with Electron UI πŸ€– πŸ’¬
Will Taner for LLMWare

Posted on

Evaluating LLMs and Prompts with Electron UI πŸ€– πŸ’¬

What is this UI useful for? 🀨

LLMs are becoming an increasingly prevalent tool across various industries. However, achieving optimal results greatly depends on selecting the correct model and prompts. This process can be extremely time-consuming as it requires extensive trial and error.

This article serves as a tutorial on a tool designed to streamline the process of testing various models and prompts. By using this tool, developers can efficiently identify the most effective combinations.

GIF

Big shoutout to Kevin Brisson for creating this Electron UI! You may find the GitHub Repo for his tool here: https://github.com/kbrisso/ai-base

Note: The CLI commands provided in this article are designed for Linux/Mac systems and may not function correctly on Windows machines. If you encounter any issues, please replace the incompatible commands with their Windows equivalents.


Step-by-Step Guide on Getting the Tool up and Running!

GIF

(If you would like to watch a video demonstrating the setup, click here)

1. Let's Start by Cloning the Repo πŸ’»

Navigate to a directory of your choosing and run git clone https://github.com/kbrisso/ai-base in your command line.

2. Root Directory Installs πŸ“¦

Navigate to the root directory, ai-base, in your command line and run npm install. If you experience an error, run npm audit fix. This will resolve any vulnerabilities in the packages.

3. llmware-wrapper Installs πŸ“¦

Navigate to the directory llmware-wrapper in your command line and run the same exact command from above: npm install and npm audit fix if there is an error.

4. Create a Virtual Environment and Install Packages πŸ’Ύ

While in the same directory as above, llmware-wrapper, create a new virtual environment: python3 -m venv venv.

Then enter into the virtual environment by running source venv/bin/activate in the command line.

Then run pip install -r requirements.txt to install the required packages.

Finally, deactivate the virtual environment by running deactivate

5. Copy Local Python Path 🐍

On the file explorer on your IDE, open the file titled llmware-wrapper.properties in the llmware-wrapper directory.

In this file, delete the path that the variable pythonpath is already set to and replace it with the path to your local python interpreter.

If you do not know what your local path is, run which python in the command line then copy and paste the result where you deleted the previous path.

GIF


Let's start the UI! πŸš€

Navigate to the root directory, ai-base, in your command line and run npm start. After a few seconds a window of the UI should pop up.

UI startup image


Selecting a Model and Prompt πŸ’‘

Click on the button that says "Choose a Model". This will display all the available models provided by LLMWare. Once you find a model that you would like to try, click the button that says "Choose" next to the model name.

models image

Click on the button that says "Choose a Prompt". This will display all the available types of prompts you can choose from provided by LLMWare. Additionally, you may find supplemental information about the prompt type, such as a description, to the right of the prompt name. Once you find a prompt type that you would like to try, click the button that says "Choose" next to the prompt name.

prompt image


Putting it all together... 🧩

After selecting a model and a prompt type, it is time to start querying! Simply add your query to the box labeled "Query" and click the button that says "Run Query". After some time, the response will show up in the box titled "Response".

Note: Depending on the prompt type chosen, an extra box for context will appear. You may use this space to provide relevant details for your query if needed.

example image

Now you can experiment with different models and queries with ease!

Conclusion 🏁

Finding the right combination of models and prompts are crucial to creating a reliable and effective LLM tool. Utilizing this UI, you can find the best combination faster than ever before!

Please check out our Github and leave a star! https://github.com/llmware-ai/llmware

Follow us on Discord here: https://discord.gg/MgRaZz2VAB

Please be sure to visit our website llmware.ai for more information and updates

Top comments (0)