DEV Community

Cover image for Combine LangChain πŸ¦œπŸ”— & Llama-Index πŸ¦™
Adheeban Manoharan
Adheeban Manoharan

Posted on

Combine LangChain πŸ¦œπŸ”— & Llama-Index πŸ¦™

In the last post of this blog series, we saw how we could use LangChain's memory classes for building a chatbot-like interface with OpenAI's GPT-3 API. In this blog, we are basically gonna use these memory classes to keep track of our previous conversations with the LLM all the while making it answer our queries regarding the contents of the document we feed to it.

We'll try to keep the dependencies on LlamaIndex to a bare minimum, because Langchain almost covers all the aspects of utilities that we would need to build this. We'll only be using the adapters available in the LlamaIndex library, because they perform better than the Langchain ones.

Now before we dive in, let me set the context for you as to what we are trying to achieve. We are trying to build an intelligent system that reads the contents of the document and should be able to answer any queries from that content. This intelligent should also be able to keep track of our past conversations and if at all any question pointing one of the previous queries, it should be able to answer that.

Remember when I told you in the last blog about completion tokens and context tokens? The maximum context window for GPT-3 is 4096 tokens. We should be able to squeeze all our context, memory, prompt and response all within the same window.

Something like this ->

Context Window - GPT3

With that out of the way, let's hustle !


Memory meets Custom Knowledge πŸ§ πŸ“”

Let's start right away with the code:

from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationBufferWindowMemory
from langchain.vectorstores import FAISS
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from llama_index import download_loader
import os

os.environ["OPENAI_API_KEY"] = 'Your API Key Here'

file_path = input('Enter the youtube link: ')

YoutubeTranscriptReader = download_loader("YoutubeTranscriptReader")
documents = YoutubeTranscriptReader().load_data(ytlinks=[file_path])

documents = [doc.to_langchain_format() for doc in documents]
text_splitter = RecursiveCharacterTextSplitter()
documents = text_splitter.split_documents(documents)

embeddings = OpenAIEmbeddings()
retriever = FAISS.from_documents(documents, embeddings).as_retriever(k=4)

llm=ChatOpenAI(temperature=1, model_name="gpt-3.5-turbo", max_tokens=2048)

memory = ConversationBufferWindowMemory(
                                    memory_key='chat_history',
                                    return_messages=True,
                                    k=6
                                  )

conversation = ConversationalRetrievalChain.from_llm(
    llm=llm, 
    retriever=retriever,
    # verbose=True, 
    memory=memory,
    max_tokens_limit=1536  
)

def chatbot(pt):
    res = conversation({'question': pt})['answer']
    return res

if __name__=='__main__':
    while True:
        print('########################################\n')
        pt = input('ASK: ')
        if pt.lower()=='end':
            break
        response = chatbot(pt)
        print('\n----------------------------------------\n')
        print('ChatGPT says: \n')
        print(response, '\n')
Enter fullscreen mode Exit fullscreen mode

You know the drill, just copy the code and paste it in a file and while running the file, install the dependencies if anything is prompted.

These are some important points, that I thought I should highlight in the above code:

FAISS is a library built my MetaAI for semantic search, that helps us with the similarity search within the context.

ConversationBufferWindowMemory is the memory that we've used, this type of memory keeps track of the last k conversations between the user and the LLM. I have given k=6.

RecursiveCharacterTextSplitter is an utility from LangChain that splits the context into chunks.

ConversationalRetrievalChain is a type of chain that aids in a conversational chatbot-like interface while also keeping the document context and memory intact.

That should give you an idea. If you have any more questions about the code, feel free to comment below.

Now, for a change, I have used the YoutubeTranscriptReader from the llama-index, this reads the transcript of the youtube link that you provide and builds the context out of it for the LLM. In the last blog we had used SimpleDirectoryReader.

Here's what I tried with Matt Walker's talk on Sleep is your superpower:

Enter the youtube link: https://youtu.be/5MuIMqhT8DM


########################################

ASK: list down the key points in the speech

----------------------------------------

ChatGPT says: 

1. Regularity in sleep timing is important to anchor sleep and improve its quantity and quality.
2. Keeping the bedroom cool aids in initiating and maintaining sleep.
3. Sleep is a non-negotiable biological necessity for human beings' health and well-being.
4. Sleep loss is a silent epidemic in industrialized nations and a significant public health challenge.
5. Sleep debt cannot be accumulated and repaid at a later point in time.
6. Getting out of bed when unable to sleep for too long establishes the association between bed and sleep.
7. The importance of prioritizing sleep for overall health and well-being. 

########################################

ASK: now convert that into a song for a 5 year old

----------------------------------------

ChatGPT says: 

Sure, here's a simple song about the importance of sleep:

Verse 1:
We all need to rest our heads
And go to sleep in our cozy beds
Regular sleep is the key
To feeling good and healthy

Chorus:
Sleep, sleep, the magic elixir
It helps us grow and gives us vigor
Don't stay up too late at night
Or you won't feel so bright

Verse 2:
Keep your room nice and cool
And your sleep will be extra smooth
And if you can't fall asleep
Get out of bed and don't make a peep

Chorus:
Sleep, sleep, the magic elixir
It helps us grow and gives us vigor
Don't stay up too late at night
Or you won't feel so bright

Verse 3:
Sleep is important, it's true
For our brains and our bodies too
We won't feel so good without it
So let's all just go to bed

Chorus:
Sleep, sleep, the magic elixir
It helps us grow and gives us vigor
Don't stay up too late at night
Or you won't feel so bright

Outro:
So let's all get a good night's sleep
And tomorrow, we'll feel so sweet
Sleep tight and have nice dreams
And wake up feeling like a new team! 

########################################
Enter fullscreen mode Exit fullscreen mode

Notice how the LLM is able to sense that I'm referring the previous conversation in the second question and converts that into a song? Exactly what we were trying to achieve. Do try it out with your own Youtube video.


Now, that's about LangChain and Llama-Index. In the next blog of this series, we'll have a look at all the awesome adapters that are available in the Llama-Hub. Cheers, see ya in the next one πŸ˜‰

Top comments (4)

Collapse
 
praveenr2998 profile image
praveenr

Wow this is awesome.

Collapse
 
mrasoahaingo profile image
Michael Rasoahaingo

Nice! did you try to use llamaindex as a retriever?

Collapse
 
amitkayal profile image
Amit Kayal

This is quite great but I am not sure why we need gpt_index here at all. We should be able to achieve all these by langchain and llm from openAI.

Collapse
 
iamadhee profile image
Adheeban Manoharan

I completely agree. But gpt_index has some pretty great adapters that is best integrated with the llamahub.ai, these adapters are also easy to use as compared to langchain ones. I have raised this concern with the Langchain team already, they are working on it.