DEV Community

Connie Leung
Connie Leung

Posted on

Build Agentic RAG application using langchain.js, nestjs, Htmx, and Gemma 2

#rag #nestjs #htmx #gemma2

In this blog post, I describe how to use Langchain, NestJS, and Gemma 2 to build an agentic RAG application. Then, the HTMX and Handlebar template engine render the responses in a list. The application uses Langchain to create a built-in DuckDuckGoSearch tool to look for information on the Internet. It also builds a custom tool to call a Dragon Ball Z API to filter characters to return their race, affiliation, and abilities. Finally, I build two retriever tools to retrieve Angular Signal and Angular Form web pages from angular.dev.

These tools bind to Gemma 2 model; and the model, tools, and chat history are passed to a Langchain agent. The agent invokes when it receives a query, and it has the intelligence to generate function call, and use the right tool to come up with a response.

Set up environment variables

Copy .env.example to .env

PORT=3001
GROQ_API_KEY=<GROQ API KEY>
GROQ_MODEL=gemma2-9b-it
GEMINI_API_KEY=<GEMINI API KEY>
GEMINI_TEXT_EMBEDDING_MODEL=text-embedding-004
SWAGGER_TITLE='Langchain Search Agent'
SWAGGER_DESCRIPTION='Use Langchain tools and agent to search information on the Internet.'
SWAGGER_VERSION='1.0'
SWAGGER_TAG='Gemma 2, Langchain.js, Agent Tools'
DUCK_DUCK_GO_MAX_RESULTS=1
Enter fullscreen mode Exit fullscreen mode

Navigate to https://aistudio.google.com/app/apikey, sign in to create a new API Key. Replace the API Key to GENINI_API_KEY.

Navigate to Groq Cloud, https://console.groq.com/, sign up and register a new API Key. Replace the API Key to GROQ_API_KEY.

Install the dependencies 

npm i -save-exact @google/generative-ai @langchain/community
@langchain/core @langchain/google-genai @langchain/groq @nestjs/axios @nestjs/config @nestjs/swagger @nestjs/throttler axios cheerio class-transformer class-validator compression duck-duck-scrape hbs langchain zod
Enter fullscreen mode Exit fullscreen mode

Define the configuration in the application

Create a src/configs folder and add a configuration.ts to it

export default () => ({
 port: parseInt(process.env.PORT || '3001', 10),
 groq: {
   apiKey: process.env.GROQ_API_KEY || '',
   model: process.env.GROQ_MODEL || 'gemma2-9b-it',
 },
 gemini: {
   apiKey: process.env.GEMINI_API_KEY || '',
   embeddingModel: process.env.GEMINI_TEXT_EMBEDDING_MODEL || 'text-embedding-004',
 },
 swagger: {
   title: process.env.SWAGGER_TITLE || '',
   description: process.env.SWAGGER_DESCRIPTION || '',
   version: process.env.SWAGGER_VERSION || '',
   tag: process.env.SWAGGER_TAG || '',
 },
 duckDuckGo: {
   maxResults: parseInt(process.env.DUCK_DUCK_GO_MAX_RESULTS || '1', 10),
 },
});
Enter fullscreen mode Exit fullscreen mode

Create a src/configs/types folder, and add duck-config.type.ts and groq-config.type.ts files. DuckDuckGoConfig and GroqConfig are configuration types that store the environment variables to the custom objects.

// duck-config.type.ts

export type DuckDuckGoConfig = {
 maxResults: number;
};
Enter fullscreen mode Exit fullscreen mode 
// groq-config.type.ts

export type GroqConfig = {
 model: string;
 apiKey: string;
};
Enter fullscreen mode Exit fullscreen mode

Create a Angular Doc Module

Create an Angular Doc module for the retriever tools that generate responses from the official documentation of Angular.

nest g mo angularDoc
Enter fullscreen mode Exit fullscreen mode

Add a embedding model

Add a Gemini Text Embedding model to calculate the documents into an array of vectors. Create a create-embedding-model.ts file under application/embeddings folder.

// application/types/embedding-model-config.type.ts

export type EmbeddingModelConfig = {
 apiKey: string;
 embeddingModel: string;
};
Enter fullscreen mode Exit fullscreen mode 
// application/embeddings/create-embedding-model.ts

import { TaskType } from '@google/generative-ai';
import { GoogleGenerativeAIEmbeddings } from '@langchain/google-genai';
import { ConfigService } from '@nestjs/config';
import { EmbeddingModelConfig } from '../types/embedding-model-config.type';

export function createTextEmbeddingModel(configService: ConfigService, title = 'Angular') {
 const { apiKey, embeddingModel: model } = configService.get<EmbeddingModelConfig>('gemini');
 return new GoogleGenerativeAIEmbeddings({
   apiKey,
   model,
   taskType: TaskType.RETRIEVAL_DOCUMENT,
   title,
 });
}
Enter fullscreen mode Exit fullscreen mode

Create documents

The helper function loads the content of a list of web pages into documents, and splits the documents into chunks. The loadWebPage is a helper function to load the web pages from angular.dev and return the split documents

// application/loaders/web-page-loader.ts

import { RecursiveCharacterTextSplitter } from '@langchain/textsplitters';
import { CheerioWebBaseLoader } from '@langchain/community/document_loaders/web/cheerio';

async function loadWebPages(webPages: string[]) {
 const loaders = webPages.map((page) => new CheerioWebBaseLoader(page));
 const docs = await Promise.all(loaders.map((loader) => loader.load()));
 const signalDocs = docs.flat();
 return splitter.splitDocuments(signalDocs);
}
Enter fullscreen mode Exit fullscreen mode

The loadSignalWebPages function loads the pages of Angular Signal into the split documents.

export async function loadSignalWebPages() {
 const webPages = [
   'https://angular.dev/guide/signals',
   'https://angular.dev/guide/signals/rxjs-interop',
   'https://angular.dev/guide/signals/inputs',
   'https://angular.dev/guide/signals/model',
   'https://angular.dev/guide/signals/queries',
   'https://angular.dev/guide/components/output-fn',
 ];

 return loadWebPages(webPages);
}
Enter fullscreen mode Exit fullscreen mode

The loadFormWebPages function loads the pages of Angular Form into the split documents.

export async function loadFormWebPages() {
 const webPages = [
   'https://angular.dev/guide/forms',
   'https://angular.dev/guide/forms/reactive-forms',
   'https://angular.dev/guide/forms/typed-forms',
   'https://angular.dev/guide/forms/template-driven-forms',
   'https://angular.dev/guide/forms/form-validation',
   'https://angular.dev/guide/forms/dynamic-forms',
 ];

 return loadWebPages(webPages);
}
Enter fullscreen mode Exit fullscreen mode

Create Retrievers

The text embedding model calculates the document chunks into vectors and the vectors are stored into MemoryVectorStore for simplicity reasons. The vector store calls asRetriever to return a vector store retriever.

private async createSignalRetriever() {
   const docs = await loadSignalWebPages();
   this.logger.log(`number of signal docs -> ${docs.length}`);
   const embeddings = createTextEmbeddingModel(this.configService, 'Angular Signal');
   const vectorStore = await MemoryVectorStore.fromDocuments(docs, embeddings);
   return vectorStore.asRetriever();
 }

 private async createFormRetriever() {
   const docs = await loadFormWebPages();
   this.logger.log(`number of form docs -> ${docs.length}`);
   const embeddings = createTextEmbeddingModel(this.configService, 'Angular Forms');
   const vectorStore = await MemoryVectorStore.fromDocuments(docs, embeddings);
   return vectorStore.asRetriever();
}
Enter fullscreen mode Exit fullscreen mode

The createSignalRetriever function returns a retriever for Angular Signal and the createFormRetriever functions returns a retriever for Angular template-driven, reactive, and dynamic forms.

Create retriever tools from the retrievers 

private async createSignalRetrieverTool(): Promise<DynamicStructuredTool<any>> {
   const retriever = await this.createSignalRetriever();
   return createRetrieverTool(retriever, {
     name: 'angular_signal_search',
     description: `Search for information about Angular Signal.
       For any questions about Angular Signal API, you must use this tool!
       Please Return the answer in markdown
       If you do not know the answer, please say you don't know.
       `,
   });
}

private async createFormRetrieverTool(): Promise<DynamicStructuredTool<any>> {
   const retriever = await this.createFormRetriever();
   return createRetrieverTool(retriever, {
     name: 'angular_form_search',
     description: `Search for information about Angular reactive, typed reactive, template-drive, and dynamic forms.
     For any questions about Angular Forms, you must use this tool!        
     Please return the answer in markdown.
     If you do not know the answer, please say you don't know.`,
   });
}

async createRetrieverTools(): Promise<DynamicStructuredTool<any>[]> {
   return Promise.all([this.createSignalRetrieverTool(), this.createFormRetrieverTool()]);
}
Enter fullscreen mode Exit fullscreen mode

The createSignalRetrieverTool function calls createRetrieverTool to create a tool from the Angular Signal retriever. The createFormRetrieverTool creates a tool from the Angular Form retriever. Finally, the createRetrieverTools function calls both createSignalRetrieverTool and createFormRetrieverTool to return an array of retriever tools.

Create an Agent module

The agent module is responsible for creating a langchain agent that executes the tools to generate responses.

nest g mo agent
nest g s agent/application/agentExecutor --flat
nest g s agent/application/dragonBall --flat
nest g s agent/presenters/http/agent --flat
Enter fullscreen mode Exit fullscreen mode

Create constants 

// agent.constant.ts

export const AGENT_EXECUTOR = 'AGENT_EXECUTOR';
Enter fullscreen mode Exit fullscreen mode 
// groq-chat-model.constant.ts

export const GROQ_CHAT_MODEL = 'GROQ_CHAT_MODEL';
Enter fullscreen mode Exit fullscreen mode 
// tools.constant.ts

export const TOOLS = 'TOOLS';
Enter fullscreen mode Exit fullscreen mode

The constants are defined to inject custom resources in the NestJS application.

Providers

The GROQ_CHAT_MODEL creates a Groq Chat Model that uses the Gemma 2 model.

// groq-chat-model.provider.ts

import { ChatGroq } from '@langchain/groq';
import { Inject, Provider } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { GroqConfig } from '~configs/types/groq-config.type';
import { GROQ_CHAT_MODEL } from '../constants/groq-chat-model.constant';

export function InjectChatModel() {
 return Inject(GROQ_CHAT_MODEL);
}

export const GroqChatModelProvider: Provider<ChatGroq> = {
 provide: GROQ_CHAT_MODEL,
 useFactory: (configService: ConfigService) => {
   const { apiKey, model } = configService.get<GroqConfig>('groq');
   return new ChatGroq({
     apiKey,
     model,
     temperature: 0.3,
     maxTokens: 2048,
     streaming: false,
   });
 },
 inject: [ConfigService],
};
Enter fullscreen mode Exit fullscreen mode

The TOOLS injects an array of tools for agent to execute to generate results.

// tool.provider.ts

import { DuckDuckGoSearch } from '@langchain/community/tools/duckduckgo_search';
import { Tool } from '@langchain/core/tools';
import { Provider } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { AngularDocsService } from '~angular-docs/application/angular-docs.service';
import { DuckDuckGoConfig } from '~configs/types/duck-config.type';
import { TOOLS } from '../constants/tools.constant';
import { DragonBallService } from '../dragon-ball.service';

export const ToolsProvider: Provider<Tool[]> = {
 provide: TOOLS,
 useFactory: async (service: ConfigService, dragonBallService: DragonBallService, docsService: AngularDocsService) => {
   const { maxResults } = service.get<DuckDuckGoConfig>('duckDuckGo');
   const duckTool = new DuckDuckGoSearch({ maxResults });
   const characterFiltertool = dragonBallService.createCharactersFilterTool();
   const retrieverTools = await docsService.createRetrieverTools();
   return [duckTool, characterFiltertool, ...retrieverTools];
 },
 inject: [ConfigService, DragonBallService, AngularDocsService],
};
Enter fullscreen mode Exit fullscreen mode

The DuckDuckGoSearch is a langchain tool to search for information on the Internet. The characterFilterTool is a custom tool that calls the Dragon Ball API to filter characters based on given criteria. The retrieverTools is an array of tools that returns knowledge of Angular Signal and Angular Form. The ToolsProvider provider returns a list of tools that the agent can execute to get the information.

import { ChatPromptTemplate } from '@langchain/core/prompts';
import { Tool } from '@langchain/core/tools';
import { ChatGroq } from '@langchain/groq';
import { Inject, Provider } from '@nestjs/common';
import { AgentExecutor, createToolCallingAgent } from 'langchain/agents';
import { AGENT_EXECUTOR } from '../constants/agent.constant';
import { GROQ_CHAT_MODEL } from '../constants/groq-chat-model.constant';
import { TOOLS } from '../constants/tools.constant';

const prompt = ChatPromptTemplate.fromMessages([
 ['system', 'You are a helpful assistant.'],
 ['placeholder', '{chat_history}'],
 ['human', '{input}'],
 ['placeholder', '{agent_scratchpad}'],
]);

export function InjectAgent() {
 return Inject(AGENT_EXECUTOR);
}

export const AgentExecutorProvider: Provider<AgentExecutor> = {
 provide: AGENT_EXECUTOR,
 useFactory: async (llm: ChatGroq, tools: Tool[]) => {
   const agent = await createToolCallingAgent({ llm, tools, prompt, streamRunnable: false });
   console.log('tools', tools);

   return AgentExecutor.fromAgentAndTools({
     agent,
     tools,
     verbose: true,
   });
 },
 inject: [GROQ_CHAT_MODEL, TOOLS],
};
Enter fullscreen mode Exit fullscreen mode

The AgentExecutorProvider provider creates an agent executor with the agent and tools. The agent executor generates the function call, and the agent calls the tools to generate the relevant responses.

Create the custom tool in the DragonBall Service 

import { DynamicStructuredTool, tool } from '@langchain/core/tools';
import { HttpService } from '@nestjs/axios';
import { Injectable } from '@nestjs/common';
import { z } from 'zod';
import { CharacterFilter } from './types/character-filter.type';
import { Character } from './types/character.type';

export const characterFilterSchema = z.object({
 name: z.string().optional().describe('Name of a Dragon Ball Z character.'),
 gender: z.enum(['Male', 'Female', 'Unknown']).optional().describe('Gender of a Dragon Ball Z caracter.'),
 race: z.enum(['Human', 'Saiyan'])
   .optional()
   .describe('Race of a Dragon Ball Z character'),
 affiliation: z.enum(['Z Fighter', 'Red Ribbon Army', 'Namekian Warrior'])
   .optional()
   .describe('Affiliation of a Dragon Ball Z character.'),
});

@Injectable()
export class DragonBallService {
 constructor(private readonly httpService: HttpService) {}

 async getCharacters(characterFilter: CharacterFilter): Promise<string> {
   const filter = this.buildFilter(characterFilter);

   if (!filter) {
     return this.generateMarkdownList([]);
   }

   const characters = await this.httpService.axiosRef
     .get<Character[]>(`https://dragonball-api.com/api/characters?${filter}`)
     .then(({ data }) => data);

   return this.generateMarkdownList(characters);
 }

 createCharactersFilterTool(): DynamicStructuredTool<any> {
   return tool(async (input: CharacterFilter): Promise<string> => this.getCharacters(input), {
     name: 'dragonBallCharacters',
     description: `Call Dragon Ball filter characters API to retrieve characters by name, race, affiliation, or gender.`,
     schema: characterFilterSchema,
   });
 }
Enter fullscreen mode Exit fullscreen mode

The getCharacters method accepts optional criteria such as name, gender, race and affiliation. Then, it appends the query parameters to the Dragon Ball URL to retrieve the characters and generate a markdown. The createCharactersFilterTool imports tool from langchain to create a custom tool that made available to the agent.

Create the Agent Executor Service 

import { AIMessage, HumanMessage } from '@langchain/core/messages';
import { Injectable } from '@nestjs/common';
import { AgentExecutor } from 'langchain/agents';
import { ToolExecutor } from './interfaces/tool.interface';
import { InjectAgent } from './providers/agent-executor.provider';
import { AgentContent } from './types/agent-content.type';

@Injectable()
export class AgentExecutorService implements ToolExecutor {
 private chatHistory = [];

 constructor(@InjectAgent() private agentExecutor: AgentExecutor) {}

 async execute(input: string): Promise<AgentContent[]> {
   const { output } = await this.agentExecutor.invoke({ input, chat_history: this.chatHistory });

   this.chatHistory = this.chatHistory.concat([new HumanMessage(input), new AIMessage(output)]);
   if (this.chatHistory.length > 10) {
     // remove the oldest Human and AI Messages
     this.chatHistory.splice(0, 2);
   }
   return [
     {
       role: 'Human',
       content: input,
     },
     {
       role: 'Assistant',
       content: output,
     },
   ];
 }
}
Enter fullscreen mode Exit fullscreen mode

The AgentExecutorService service is straightforward. It injects an instance of AgentExecutor, invokes the invoke method to submit the input to the chain and outputs a string. This method stores the Human and AI messages in chat history in memory and returns the conversation to the template engine for rendering.

private chatHistory = [];
if (this.chatHistory.length > 10) {
   // remove the oldest Human and AI Messages
   this.chatHistory.splice(0, 2);
}
Enter fullscreen mode Exit fullscreen mode

Add Agent Controller 

import { IsNotEmpty, IsString } from 'class-validator';

export class AskDto {
 @IsString()
 @IsNotEmpty()
 query: string;
}
Enter fullscreen mode Exit fullscreen mode 
@Post()
 async ask(@Body() dto: AskDto): Promise<string> {
   const contents = await this.service.execute(dto.query);
   return toDivRows(contents);
 }
Enter fullscreen mode Exit fullscreen mode

The Agent controller submits the query to the chain, gets the results, and sends the HTML codes back to the template engine to render.

Modify the App Controller to render handlebar template 

@Controller()
export class AppController {
 @Render('index')
 @Get()
 async getHello(): Promise<Record<string, string>> {
   return {
     title: 'Langchain Search Agent',
   };
 }
}
Enter fullscreen mode Exit fullscreen mode

The App controller informs the Handlebar template engine to render index.hbs file.

HTMX and Handlebar Template Engine

This is a simple user interface to display the conversation

default.hbs
<!DOCTYPE html>
<html lang="en">
 <head>
   <meta charset="utf-8" />
   <meta name="description" content="Angular tech book RAG powed by gemma 2 LLM." />
   <meta name="author" content="Connie Leung" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
   <title>{{{ title }}}</title>
   <style>
     *, *::before, *::after {
         padding: 0;
         margin: 0;
         box-sizing: border-box;
     }
   </style>
   <script src="https://cdn.tailwindcss.com?plugins=forms,typography"></script>
 </head>
 <body class="p-4 w-screen h-screen min-h-full">
   <script src="https://unpkg.com/htmx.org@2.0.1" integrity="sha384-QWGpdj554B4ETpJJC9z+ZHJcA/i59TyjxEPXiiUgN2WmTyV5OEZWCD6gQhgkdpB/" crossorigin="anonymous"></script>
   <div class="h-full grid grid-rows-[70px_1fr_40px] grid-cols-[1fr]">
     {{> header }}
     {{{ body }}}
     {{> footer }}
   </div>
 </body>
</html>
Enter fullscreen mode Exit fullscreen mode

The above is a default layout with a header, footer, and body. The body eventually displays the conversation between the AI and human. The head section import tailwind to style the HTML elements and htmx to interact with the server.

<div>
   <div class="mb-2 p-1 border border-solid border-[#464646] rounded-lg">
       <p class="text-[1.25rem] mb-2 text-[#464646] underline">Architecture</p>
       <ul id="architecture" hx-trigger="load" hx-get="/agent/architecture"
           hx-target="#architecture" hx-swap="innerHTML"></ul>
   </div>
   <div id="results" class="mb-4 h-[300px] overflow-y-auto overflow-x-auto"></div>
   <form id="rag-form" hx-post="/agent" hx-target="#results" hx-swap="beforeend swap:1s">
       <div>
           <label>
               <span class="text-[1rem] mr-1 w-1/5 mb-2 text-[#464646]">Question: </span>
               <input type="text" name="query" class="mb-4 w-4/5 rounded-md p-2"
                   placeholder="Ask the agent"
                   aria-placeholder="Placeholder to ask any question to the agent"></input>
           </label>
       </div>
       <button type="submit" class="bg-blue-500 hover:bg-blue-700 text-white p-2 text-[1rem] flex justify-center items-center rounded-lg">
           <span class="mr-1">Send</span><img class="w-4 h-4 htmx-indicator" src="/images/spinner.gif">
       </button>
   </form>
</div>
Enter fullscreen mode Exit fullscreen mode

A user can input the question in the text field and click the Send button. The button makes a POST request to /agent and appends the conversation to the list.

This is the end of my first langchain agentic RAG application using Gemma 2 model and various tools to generate the responses.

Resources

Top comments (0)

Read next

sarvabharan profile image

Promise.withResolvers: Simplifying Promise Creation in ES2024

Sarva Bharan -

baweaver profile image

Let's Read - Eloquent Ruby - Ch 16

Brandon Weaver -

techielass profile image

Enable Microsoft 365 Backups

Sarah Lean 🏴󠁧󠁢 -

odyash profile image

Tokens vs Chunks

OdyAsh -