DEV Community

Cover image for โญ 9 Open Source tools to take your AI app to the next level ๐Ÿš€๐Ÿค–
Vince Lwt for Lunary

Posted on • Updated on • Originally published at lunary.ai

โญ 9 Open Source tools to take your AI app to the next level ๐Ÿš€๐Ÿค–

The AI ecosystem is booming.

If you've started building AI-powered apps, you might have missed some of those awesome open-source projects that can help you make your LLM queries, more relevant and improve the general quality of your chatbots and AI apps.

Here are 9 projects to take your app to the next level in 2023.

go

LLMonitor (sponsored)

llmonitor

LLMonitor is an all-in-one open-source toolkit for AI devs ready to take their app to production, with features such as:

  • ๐Ÿ’ต Cost & latency analytics
  • ๐Ÿ‘ช Users tracking
  • ๐Ÿ› Traces to debug easily
  • ๐Ÿ” Inspect & replay AI requests
  • ๐Ÿท๏ธ Label and export fine-tuning datasets
  • ๐Ÿ–ฒ๏ธ Collect feedback from users
  • ๐Ÿงช Evaluate prompts against tests

Once you're ready for users to try your app, using an observability solution is essential.

We invite you to give LLMonitor a shot (it's completely free up to 1000 events / day).

โญ Give LLMonitor a โญ๏ธ on GitHub


Guidance

Guidance is a new language format released by Microsoft allowing to create complex agent flows. It looks like this:

experts = guidance('''
{{#system~}}
You are a helpful and terse assistant.
{{~/system}}

{{#user~}}
I want a response to the following question:
{{query}}
Name 3 world-class experts (past or present) who would be great at answering this?
Don't answer the question yet.
{{~/user}}

{{#assistant~}}
{{gen 'expert_names' temperature=0 max_tokens=300}}
{{~/assistant}}

{{#user~}}
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
{{~/user}}

{{#assistant~}}
{{gen 'answer' temperature=0 max_tokens=500}}
{{~/assistant}}
''', llm=gpt4)
Enter fullscreen mode Exit fullscreen mode

This interleaving of generation and prompting allows for precise output structure that might help produce clear and parsable results.

โญ Check out on GitHub


LiteLLM

Call any LLM API using the OpenAI format (Bedrock, Huggingface, Cohere, TogetherAI, Azure, OpenAI, etc.)

For example, integrating Anthropic will look like this:

from litellm import completion
import os

## set ENV variables
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"

response = completion(
  model="claude-2", 
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)
Enter fullscreen mode Exit fullscreen mode

This way, you can integrate different models in your app without learning and intergrating new APIs.

โญ Check out on GitHub


Zep

Zep

Zep allows you to summarize, embed, and enhance chat histories and documents asynchronously. It ensures that these operations do not impact the chat experience of your users.

With Zep, chatbot histories are persisted to a database, enabling you to easily scale out as your user base grows.

As a drop-in replacement for popular LangChain components, Zep allows you to get your application into production within minutes, without the need to rewrite your existing code.

โญ Check out on GitHub


LangChain

Who hasn't heard of LangChain by the now? LangChain is the most popular AI framework allowing you to plug together models into chains, with vector stores and more to build powerful AI apps.

โญ Check out on GitHub


DeepEval

DeepEval

DeepEval is an evaluation framework designed for LLM apps that is straightforward to use and available open-source.

It functions similarly to Pytest, but with a specialization in testing LLM applications. DeepEval assesses using metrics like factual consistency, accuracy, and answer relevancy utilizing LLMs along with various other NLP models.

โญ Check out on GitHub


pgVector

pgVector is a Postgres extension to store your embeddings and perform operations such as similarity search.

If you're using Supabase, pgVector is already available.

You could use pgVector instead of specialized vector databases Pinecone to simplify your stack.

โญ Check out on GitHub


promptfoo

promptfoo

With promptfoo, similarly to deepEval, you can test your prompts & models against predefined test cases.

Evaluate quality and catch regressions by comparing LLM outputs side-by-side, score outputs automatically by defining test cases.

โญ Check out on GitHub


Model Fusion

Model Fusion is a TypeScript library designed for building AI applications, chatbots, and agents.

It offers support for a wide range of models, including text generation, image generation, text-to-speech, speech-to-text, and embedding models.

Features:

  • Multimodal: combine different modalities such as text, images, and speech.
  • Streaming: Model Fusion supports streaming for many generation models, including text streaming, structure streaming, and full duplex speech streaming.
  • Utility: Model Fusion provides a set of utility functions for tools and tool usage, vector indices, and guard functions.
  • Type inference and validation: Model Fusion leverages TypeScript to infer types and validate model responses.

Useful if you're prefer Typescript to Python.

Model fusion is a quite new but very promising project.

โญ Check out on GitHub


Thank you for reading!

Any project we missed? please tell us in the comments :)

A star on our GitHub project would mean the world ๐Ÿ™ Click on the cat to make him happy ๐Ÿ‘‡

Stars please

Top comments (7)

Collapse
 
annaredbond profile image
annaredbond

Great write-up!

Collapse
 
nevodavid profile image
Nevo David

Great tools for using AI!

Collapse
 
vincelwt profile image
Vince Lwt

Thank you Nevo!

Collapse
 
fernandezbaptiste profile image
Bap

Great piece as usual!

Collapse
 
hugues_chocart_72be90620d profile image
Hugues Chocart

Thanks for sharing! Excited to explore these tools in my next AI project

Collapse
 
vincelwt profile image
Vince Lwt

Glad to hear!

Collapse
 
skeptrune profile image
Nick K

Great resources!