Github repository at the end of the article
So you coded a cool Langchain agent in Python. Congrats! In this tutorial, we will walk you through the process of making it an OpenAPI endpoint, which can be deployed and called as an API, allowing you to seamlessly integrate it into your product or workflows. All that in only 10 minutes. Let’s get started!
Installation and setup
You need to have Python 3.8 or higher installed.
(Optional) Create a new virtual environment
python3 -m venv venv
source venv/bin/activate
Install the required packages
-
langchain
for the orchestration of the agent -
openai
for the OpenAI API -
phospho
for the deployment of the agent -
python-dotenv
for the environment variables (like api keys)
pip install --upgrade langchain openai phospho python-dotenv
pip freeze > requirements.txt
Store your OpenAI API key
In this example, we are going to use OpenAI as our LLM provider for our Langchain agent. But you can use any LLM provider you wish.
OPENAI_API_KEY=your-api-key
Create an .env
file and store your OpenAI API key in it. For extra security, you can create a new OpenAI key for this project.
Build the agent logic
Create a new langchain agent
Create a main.py
python file at the route of the project. You must name it main.py
since phospho will look for this file to initialize the agent.
Let’s load the environment variables from the .env
file :
# main.py
# Load environment variables
from dotenv import load_dotenv
load_dotenv()
Now, let’s create a basic chatbot using the OpenAI API. As you can imagine, the process would be roughly the same for more complex agents.
# main.py
# Load environment variables
from dotenv import load_dotenv
load_dotenv()
# use langchain to easily interact with the OpenAI API
from langchain.schema import (
AIMessage,
HumanMessage,
SystemMessage
)
from langchain.chat_models import ChatOpenAI
chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)
messages = [
SystemMessage(content="You are a funny and helpful assistant."),
HumanMessage(content="Explain to me quantum physics."),
]
response=chat(messages)
print(response.content,end='\n')
Try it locally :
python main.py
Now, let’s package this logic into a function that we can use in our agent :
# main.py
# Load environment variables
from dotenv import load_dotenv
load_dotenv()
# import schema for chat messages and ChatOpenAI in order to query chatmodels GPT-3.5-turbo or GPT-4
from langchain.schema import (
AIMessage,
HumanMessage,
SystemMessage
)
from langchain.chat_models import ChatOpenAI
chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)
def my_agent(input):
messages = [
SystemMessage(content="You are a funny and helpful assistant."),
HumanMessage(content=input),
]
response=chat(messages)
return response.content
Deploy the agent
Having the agent run locally is cool. But now, let’s deploy it so everyone can use it.
To do so, we are going to package our langchain agent into a phospho agent. phospho is an agnostic Plateform as a Service to deploy and manage autonomous agents. They also provide an SDK to help developers go from prototype to production with their agents.
Here, we only want to create a basic conversational agent. So we only need to create a chat route.
# main.py
from langchain.schema import (
AIMessage,
HumanMessage,
SystemMessage
)
from langchain.chat_models import ChatOpenAI
from phospho import Agent, Message
# Load environment variables
from dotenv import load_dotenv
load_dotenv()
# Define our langchain agent
chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)
def my_agent(input):
messages = [
SystemMessage(content="You are a funny and helpful assistant."),
HumanMessage(content=input),
]
response=chat(messages)
return response.content
# Package it into a phospho agent
agent = Agent()
@agent.chat()
def my_chat(message):
# Use our agent logic to generate a response
response = my_agent(message.content)
# Return the response in a Message object
return Message(response)
That’s it!
With this code, we have created a new agent with a single route /chat
that will call our my_agent
function and send the response back to the user. Note that you need to send an object of type Message
as a response (view the documentation for more info).
Create a new Github repository
At the root of your project, create a new Github repository and push your code to the main
branch. Do not forget to add a .gitignore
file to ignore the .env
file (there is your API key inside) and the venv
folder :
Create a new phospho project
Go to your phospho account dashboard and create a new project.
For the project name :
- respect naming conventions, use just alphanumeric characters (no spaces, no special characters, no emojis, etc…)
- if the name is already taken, choose a new one. Come on, be creative!
Select the github repository you want to link to your phospho project.
Setup the environment variables
Remember, we have stored our OpenAI API key in a .env
file. But this file is not available on the server since we didn't push it to Github. Our code won't work if the API key is not available as an environment variable to the langchain agent.
On the phospho webapp, go to your project page and click on the Manage Env Variables tab. Then, click on the Environment variables tab and add a new variable called OPENAI_API_KEY
with your OpenAI API key as a value.
At the next restart of your agent, it will be able to use the OpenAI API (to force a restart, you can trigger a new deployment).
Deploy the agent
Once project is created, just push some code to the main
branch or go to the project page and trigger the deployment manually using the button.
After a one or two minutes, your agent should be deployed and ready to use. You have the endpoint URL on your project page.
Test your agent
Your agent is now deployed as an OpenAPI endpoint. You can find the URL of the API endpoint of your agent on the project page. You can test it directly using the API or by going in your browser to the URL and adding /docs
at the end.
Unless you go viral, you should be able to use the free plan (no credit card required to start).
Github repo with the source code
You can find all the source code of this tutorial in this github repo
Congratulations! You have successfully deployed a simple conversational agent using Langchain, OpenAI and phospho. Feel free to explore and enhance the capabilities of your agent further.
Want to give anonymous feedback? Fill in the google form here.
Top comments (0)