When your prompt is too large and the LLM starts to hallucinate, or when the data you want from the LLM is too extensive to be handled in one response, asynchronous calling can help you get the desired output. In this brief blog, I will teach you how to call Gemini Pro asynchronously to achieve the best results.
let's go
First create project in new directory then install following Package
pip install asyncio
pip install python-dotenv
pip install aiohttp
Go to this following link to get gemini api key
https://aistudio.google.com/app/apikey
in the above picture you can see that *create API key * click on it and get your api key
Now, create a file named .env in your project directory and add your key like this:
API_KEY = "here you copy paste the API key "
Next, create a file named main.py in your project directory.
Let's start coding the** main.py** file. First, import all necessary libraries and retrieve the API key from the .env file.
import asyncio
import os
from dotenv import load_dotenv
import aiohttp
load_dotenv()
API_KEY = os.getenv("API_KEY")
Now create a async function which return all list of all prompts
async def prompts() -> list:
heros = """
Give me the list of all 10 top highest win rate Dota 2 hero 2023
"""
players = """
Top players in the game Dota 2 in 2023
"""
team = """
Give me the name the name of all team who got directe invite in TI 2023 dota 2.
"""
return [heros, players, team ]
Now we are going to send an asynchronous POST request to the Google Generative Language API to generate content based on a given prompt. To do this, we will first set the endpoint we are going to access, then define the headers, and finally create the payload with all the necessary parameters for the endpoint. After that, we will send an asynchronous call to the endpoint. We are accessing response in json.
Note: Session (aiohttp.ClientSession): An instance of the aiohttp ClientSession class.
async def fetch_ai_response(session, prompt):
url = f"https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?key={API_KEY}"
headers = {
"Content-Type": "application/json"
}
payload = {
"contents": [
{
"parts": [
{
"text": prompt
}
]
}
]
}
async with session.post(url, headers=headers, json=payload) as response:
result = await response.json()
# Extract text from the response
try:
content = result['candidates'][0]['content']['parts'][0]['text']
return content
except (KeyError, IndexError) as e:
# Log the error and response for debugging
print(f"Error parsing response: {e}")
print(f"Unexpected response format: {result}")
return "Error: Unexpected response format"
following function takes a list of prompts and uses the fetch_ai_response function to retrieve the AI response for each prompt.
The function then uses asyncio.gather to run the fetch_ai_response function in parallel for each prompt.
The results are returned as a list of responses.
async def test_questions_from_ai() -> list:
prompts_list = await prompts()
async with aiohttp.ClientSession() as session:
tasks = [fetch_ai_response(session, prompt) for prompt in prompts_list]
results = await asyncio.gather(*tasks)
return results
Now calling test_questions_from_ai function Asynchronously
if __name__ == "__main__":
responses = asyncio.run(test_questions_from_ai())
for inx, response in enumerate(responses):
print(f"Response: {inx} ", response)
now run following command to run see the response
python main.py
check the code on following repo on github
https://github.com/GoAndPyMasters/asyncgemini
here is my github profile
https://github.com/MuhammadNizamani
If you need any help contact with on linkedin
https://www.linkedin.com/in/muhammad-ishaque-nizamani-109a13194/
Top comments (0)