Skipping to the Good Parts: Implementing Netflix's Skip Intro Feature with Python Video Processing

Netflix’s Skip Intro feature is about as iconic as the service itself, allowing subscribers to skip past intro and outro portions of tv shows. Since then, other streaming services have adopted this feature, making it a staple of the streaming world.

Recently, I began thinking about implementing this feature and this led me to make a small library to find similar portions in videos. It does this by hashing videos into frames, hashing these frames, and comparing similar frames between each comparison video and a reference video.

I decided to test this library on my favourite recently-cancelled Netflix show, Inside Job:

# Import the library
from similar_vid.similar_secs import Similar, save

# Load the file paths
file1 = "videos//inside_job//01.mkv"
file2 = "videos//inside_job//02.mkv"
file3 = "videos//inside_job//03.mkv"
file4 = "videos//inside_job//04.mkv"
file5 = "videos//inside_job//05.mkv"


if __name__ == "__main__":
    # Hash the videos
    inside_job = Similar(ref=file1, comp_arr = [file2, file3, file4, file5])

    # Match the videos
    inside_job.match(threshold=15)

    for match in inside_job.matches:
        print(match)

    # use the library's save function to save class fields as JSON for further use
    save(inside_job.matches, "outs//matches.json")

In the above, we import the Similar class from my similar-vid library and pass it the reference video and an array of comparison videos. We then use the match method to speficy that we only want matches that are at least 15 seconds long. The library then hashes and matches the videos as such:

Description: A screenshot of the console showing the hashing and matching process.

These hashes are then stored in the matches field of the class instance as a list of dictionaries. Below that, we print the matches and use the library’s save method to save the matches to a json file:

The returned arrays of the “matches” key, if any, are the beginning and ending matching seconds of the comparison video, respectively.

We can then pass this as context to the frontend. Using Javascript to modify the video’s currentTime property, we can skip the intro portion:

As can be seen in the above gif, the Skip Intro button appears over the intro portion.

The similar_vid library can be found here: https://github.com/jahwi/similar-vid

Thanks for reading.