NeoChat: A Context-Aware RAG Chatbot Starter kit

This is a submission for the Neon Open Source Starter Kit Challenge : Ultimate Starter Kit

My Kit

This starter kit provides a foundation for building a Vue-based chatbot utilizing Retrieval Augmented Generation (RAG). It combines Nuxt, Neon, and OpenAI to create a powerful, context-aware chatbot application. The kit offers state management using Pinia, styling with NuxtUI, and a minimal chat interface integrated with Neon database as a vector store.

Why Use This Starter Kit?

• Ideal for developers looking to build AI chatbot applications using Vue.js
• Perfect for implementing Retrieval Augmented Generation in scenario-specific chatbots

Key Features

• RAG Architecture: Implements a Retrieval-Augmented Generation system for accurate, context-aware responses
• Conversation History: Stores chat history for enhanced context and continuity
• Neon Postgres Integration: Utilizes Neon Postgres as a vector database for efficient embedding storage and retrieval
• OpenAI Integration: Leverages OpenAI's powerful language models for response generation
• Nuxt 3 Framework: Built on Nuxt 3, offering a modern, performant foundation for web applications
• Tailwind CSS: Utilizes Tailwind CSS for rapid UI development and easy customization
• Modular Design: Structured for easy understanding and extension of core functionalities
• Serverless-Ready: Prepared for deployment on serverless platforms like Cloudflare, Netlify and Vercel.

Link to Kit

Live Demo: NeoChat Demo
Repository: NeoChat Starter Kit

Architecture Overview

• Frontend: Nuxt 3 application with Tailwind CSS for styling
• Backend: Nuxt server routes for handling API requests
• Vector Database: Neon Postgres with pgvector for storing and querying document embeddings
• Language Model: OpenAI's gpt-4o-mini model for generating responses and text-embeddings-3-small for creating vector embeddings in 1536 dimensions.
• RAG Pipeline:
  • Query embedding
  • Vector similarity search
  • Context retrieval
  • Response generation

The Development Journey

Technology Stack Rationale

The choice of technologies for this starter kit was driven by a desire to create a developer-friendly, efficient, and scalable solution for AI-powered chatbots.

1. Nuxt: Selected for its ability to create Vue-based fullstack applications with Server Side Rendering. Vue's developer-friendly nature encourages more developers to explore AI application development.

2. OpenAI: Chosen for both the chatbot model and embeddings generation due to its developer-friendly interface and robust capabilities in creating embeddings and developing applications.

3. Neon DB: Opted for as the serverless Postgres database solution, eliminating the need for manual configuration and autoscaling concerns. Its native support for vector embeddings using PGVector and intuitive developer dashboard significantly enhance the development experience.

4. Langchain: Utilized to seamlessly integrate OpenAI and NeonDB for text loading, embedding generation, and Retrieval Augmented Generation (RAG) implementation.

Challenges and Solutions

1. Optimizing Vector Embeddings

   • Challenge: Determining the optimal chunk size and overlap for creating effective vector embeddings.
   • Solution: Extensive experimentation to find the right balance, ensuring the AI model had sufficient information to create meaningful embeddings.

2. Enhancing Context Awareness

   • Challenge: Creating a chatbot capable of providing rich, contextual answers beyond simple similarity searches.
   • Solution: Development of features to extract standalone questions, retrieve relevant embeddings, and synthesize information using an LLM. This process involved careful prompt engineering to achieve desired results.

3. Implementing Conversation History

   • Challenge: Enabling the chatbot to make context-aware responses by referencing earlier messages.
   • Solution: Implementation of an in-memory array to store conversation history, leveraging Langchain to determine the appropriate sequence of prompts and calls.

4. Frontend Development

   • Challenge: Creating an intuitive user interface for chatbot interactions.
   • Solution: Utilization of Vue and Nuxt as a meta-framework, allowing for the development of both frontend and backend components within a single framework. This approach simplified the build process and reduced configuration overhead.

5. Testing and Deployment

   • Challenge: Resolving package dependency issues during the build and deployment phase.
   • Solution: Transition to a cloud-based editor to streamline the build process, followed by successful deployment on Vercel.

The Value of Starter Kits

This starter kit aims to simplify the process of creating context-aware chatbots by providing a ready-to-use project structure. It allows developers to focus on building and shipping features without the need to manually configure all components from scratch.

Much like how Neon DB streamlines database integration for fullstack applications, this starter kit enables developers to concentrate on creating and deploying innovative features in AI-powered chat applications.

Open for Contributions

This project welcomes contributions from the community. Developers are encouraged to raise issues, request features, or submit pull requests to enhance this starter template. It serves as a foundation for building exciting new projects in the realm of AI-powered chat applications.

Acknowledgments

• The RAG logic implementation was greatly influenced by The Official Langchain Course at Scrimba.
• The comprehensive documentation provided by Neon, Nuxt, and Langchain was instrumental in overcoming challenges and finding solutions throughout the development process.