Built Perplexity AI with NextJS and Open Source LLMs

Demo

https://heysensei.app

Introduction

Recently, I embarked on a journey to build an open-source Perplexity AI using NextJS and open-source Large Language Models (LLMs). This project combined the power of modern web development with the capabilities of state-of-the-art AI models, aiming to create a versatile, efficient, and user-friendly application. Here's a detailed look at the development side of things.

Project Overview

The project, named "Sensei," can be found on GitHub. It leverages NextJS for the frontend and open-source LLMs for natural language processing. The main goal was to build a Perplexity AI, a search data-based Retrieval-Augmented Generation (RAG) agent, using completely open-source technologies.

Why NextJS?

NextJS was a natural choice for this project due to its robust features, including server-side rendering, static site generation, and API routes. These features provided the flexibility and performance needed to handle the dynamic interactions and real-time data processing required by the AI components.

Tailwind CSS and shadcn for Styling

One of my key decisions was to avoid using a traditional component library and instead build the UI with Tailwind CSS and shadcn. Here’s why this combination turned out to be a productive choice:

• Utility-First Approach: Tailwind's utility-first approach allowed for rapid prototyping and easy adjustments, making the development process more efficient.
• Customizability: Tailwind provided the flexibility to create custom styles without being constrained by predefined components.
• Component-Based Development: shadcn offered a set of highly customizable and accessible components, making it easier to maintain consistency and build a polished UI.
• Responsive Design: Built-in responsive design utilities helped in creating a seamless experience across different devices.

Building the Frontend

The frontend of the application focused on creating an intuitive user interface that facilitates seamless interaction with the AI.

Flow Engineering Over Function Calling

Instead of relying on function calling, the application leverages flow engineering. This approach simplifies the interaction between the frontend and the AI models, reducing complexity and improving performance. The decision to use flow engineering was driven by the need to handle long RAG prompts effectively.

Learnings and Challenges

1. Context Window Length: Handling long context windows was challenging but crucial for providing accurate responses. Ensuring the AI could process large amounts of data without losing context was a key focus.
2. Instruction Following: Many open-source models struggled with following complex instructions. Prompt engineering and extensive testing were necessary to achieve desired results.
3. Mix of Agents: Using a mix of lighter and heavier models helped reduce the Time to First Byte (TTFB), but it also introduced challenges related to language support and consistency in responses.

Conclusion

Building Perplexity AI with NextJS and open-source LLMs was a rewarding experience. The combination of modern web development techniques and advanced AI capabilities resulted in a powerful and flexible application. Tailwind CSS and shadcn proved to be an excellent choice for styling, enabling rapid development and a responsive design.

If you're interested in the project, you can check it out on GitHub. I'm excited to continue improving it and exploring more ways to integrate open-source technologies in meaningful ways.

Feel free to reach out with any questions or feedback. Happy coding!