DEV Community

Daniel San
Daniel San

Posted on


YoutubeGPT, start a chat with a video

Image description

With Youtube GPT you will be able to extract all the information from a video on YouTube just by pasting the video link.
You will obtain the transcription, the embedding of each segment and also ask questions to the video through a chat.

All code was written with the help of Code GPT

Captura de Pantalla 2023-02-08 a la(s) 9 16 43 p m


  • Video transcription with OpenAI Whisper
  • Embedding Transcript Segments with the OpenAI API (text-embedding-ada-002)
  • Chat with the video using streamlit-chat and OpenAI API (text-davinci-003)


For this example we are going to use this video from The PyCoach

Add the video URL and then click Start Analysis

Pytube and OpenAI Whisper

The video will be downloaded with pytube and then OpenAI Whisper will take care of transcribing and segmenting the video.
Pyyube Whisper

# Get the video 
youtube_video = YouTube(youtube_link)
streams = youtube_video.streams.filter(only_audio=True)
mp4_video ='youtube_video.mp4')
audio_file = open(mp4_video, 'rb')

# whisper load base model
model = whisper.load_model('base')

# Whisper transcription
output = model.transcribe("youtube_video.mp4")
Enter fullscreen mode Exit fullscreen mode

Embedding with "text-embedding-ada-002"

We obtain the vectors with text-embedding-ada-002 of each segment delivered by whisper

# Embeddings
segments = output['segments']
for segment in segments:
    openai.api_key = user_secret
    response = openai.Embedding.create(
        input= segment["text"].strip(),
    embeddings = response['data'][0]['embedding']
    meta = {
        "text": segment["text"].strip(),
        "start": segment['start'],
        "end": segment['end'],
        "embedding": embeddings
Enter fullscreen mode Exit fullscreen mode

OpenAI GPT-3

We make a question to the vectorized text, we do the search of the context and then we send the prompt with the context to the model "text-davinci-003"


We can even ask direct questions about what happened in the video. For example, here we ask about how long the exercise with Numpy that Pycoach did in the video took.


Running Locally


  1. Clone the repository
git clone
cd youtube-gpt
Enter fullscreen mode Exit fullscreen mode
  1. Install dependencies

These dependencies are required to install with the requirements.txt file:

pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode
  1. Run the Streamlit server
streamlit run
Enter fullscreen mode Exit fullscreen mode

Upcoming Features

  • Semantic search with embedding
  • Chart with emotional analysis
  • Connect with Pinecone

Top comments (0)