DEV Community

Timotej Avsec
Timotej Avsec

Posted on

Video content search using MongoDB Atlas Search and Google Machine Learning

What I built

I’ve built an application in Golang, which is able to find video content from all of the videos that have been uploaded to the application. It utilises the Google Cloud Video Intelligence API to detect the features (content) inside the video.

Category Submission:

I decided to submit my application into multiple categories. Below is a list of all of the categories that I would like to submit my application in, as well as description as why I think the app can be submited into listed category.

  • Search No More: Main feature of the application is to perform a quick search on ALL of the uploaded videos. That’s why I used Atlas Search, as it provides quick setup and is querying super fast.
  • Think Outside the JS Box: As an backend developer, it was my goal for quite some time now to learn Golang, but I was procrastinating with learning Go, that’s why decided that the best way to learn new programming language is to build something with it. That’s why I picked Golang as my API language.
  • Google Cloud Superstar: The “brains” behind the Video Feature detection Google Cloud Video Intelligence API, which provides Golang SDK and enables the developer to utilise powerful Google Cloud ML engine to detect the video features.

App Link

Because I use Google Cloud Video Intelligence API, which is not free, I did not want to post my application publicly and create some “unwanted” costs on my Google Cloud bill 😃. However, I recorded a short demo video, as well as screenshoted the application. After all, if you would like to try my app, you can always clone Git repo, add your Google Cloud credentials, and run the API.

Video

Screenshots

search page

video upload modal

Background

My main motivation to build this application was to learn a few things:

  • learn how to implement MongoDB into application using SDKs and libraries, and also connect Atlas Search
  • learn how Google Cloud Video Intelligence works
  • learn new programming language (Golang)

How I built it

I started my project with developing API server in Golang. For the HTTP library I used Gin framework, as it provides an friendly interface to build REST ready API.

For my application, two endpoints were enough. One was used for video upload, and one for video search. This is how endpoints are defined in Gin:

func main() {
    r := gin.Default()
    r.POST("/videos", Controllers.VideoStore)
    r.GET("/videos", Controllers.VideoSearch)

    err := r.Run()
    if err != nil {
        log.Fatalf(err.Error())
        return
    }

    fmt.Println("Server running on :8080")
}
Enter fullscreen mode Exit fullscreen mode

Video upload and feature detection

Video store method consists of the folowing steps:

File upload and transfer to Google Storage Buckets

User can upload the video to our API. Our API then uploads it to the Google Storage Bucket. This is the code that accomplishes that:

// ...
// Client init, ...
// ...

f, uploadedFile, _ := c.Request.FormFile("video")

    if filepath.Ext(uploadedFile.Filename) != ".mp4" {
        c.JSON(http.StatusUnprocessableEntity, gin.H{
            "error": "Uploaded file must be mp4 video",
        })
        return
    }

  // Upload file to Storage Bucket
    sw := Services.StorageBucket.Object(fileUuid.String()).NewWriter(ctx)

    _, err = io.Copy(sw, f)
    err = sw.Close()
    if err != nil {
        return
    }
Enter fullscreen mode Exit fullscreen mode

Uploaded file is added as an record to the MongoDB database, so we can reference it later. We save video file name, size and unique UUID, which we can reference when retrieving the private object.

// ...

coll := Services.MongoClient.Database(os.Getenv("MONGO_DB_DATABASE")).Collection("videos")
    doc := bson.D{{"filename", uploadedFile.Filename}, {"size", uploadedFile.Size}, {"uuid", fileUuid.String()}}
    mongoRecord, err := coll.InsertOne(context.TODO(), doc)

    if err != nil {
        panic(err)
    }
Enter fullscreen mode Exit fullscreen mode

3.File can be sent to Google Video Intelligence API, and retrieve the video features.

// ...
op, err := client.AnnotateVideo(ctx, &videopb.AnnotateVideoRequest{
        InputUri: "gs://" + os.Getenv("GOOGLE_CLOUD_STORAGE_BUCKET") + "/" + fileUuid.String(),
        Features: []videopb.Feature{
            videopb.Feature_LABEL_DETECTION,
        },
    })
    if err != nil {
        log.Fatalf("Failed to start annotation job: %v", err)
    }

    resp, err := op.Wait(ctx)
    if err != nil {
        log.Fatalf("Failed to annotate: %v", err)
    }
    result := resp.GetAnnotationResults()[0]
Enter fullscreen mode Exit fullscreen mode

Retrieved features are saved to the corresponding video entity in MongoDB database, where Atlas search can index them.

// ...
filter := bson.D{{"_id", mongoRecord.InsertedID}}
    update := bson.D{{"$set", bson.D{{"features", featuresList}}}}
    _, err = coll.UpdateOne(context.TODO(), filter, update)
    if err != nil {
        return
    }
Enter fullscreen mode Exit fullscreen mode

After that, the video upload process is complete. If you would like to see the whole method, and not only the code snippets, you can access it on the GitHub: https://github.com/tavsec/devto-mongodb-hackathon-api/blob/main/Controllers/VideoController.go

Logic flow

Video search

If user wants to search all of the keywords, there is a VideoSearch method for that. The steps to utilise the Atlas Search engine is the following:

type SearchBody struct {
    Keyword string `form:"keyword"`
    Page    int64  `form:"page"`
    PerPage int64  `form:"perPage"`
}

type PaginatedResult struct {
    Videos     []Video
    Pagination pag.PaginationData
}

func VideoSearch(c *gin.Context) {
    var searchBody SearchBody
    err := c.BindQuery(&searchBody)
    if err != nil {
        println(err.Error())
        err = c.AbortWithError(http.StatusBadRequest, err)
        return
    }

    collection := Services.MongoClient.Database(os.Getenv("MONGO_DB_DATABASE")).Collection("videos")

    searchStage := bson.M{"$search": bson.D{{"index", os.Getenv("MONGO_DB_SEARCH_INDEX")}, {"text", bson.D{{"path", bson.D{{"wildcard", "*"}}}, {"query", searchBody.Keyword}}}}}

    aggPaginatedData, err := pag.New(collection).Context(context.TODO()).Limit(searchBody.PerPage).Page(searchBody.Page).Aggregate(searchStage)
    if err != nil {
        panic(err)
    }

    storageClient, _ := storage.NewClient(context.Background())

    opts := &storage.SignedURLOptions{
        Scheme:  storage.SigningSchemeV4,
        Method:  "GET",
        Expires: time.Now().Add(15 * time.Minute),
    }

    var results []Video
    for _, raw := range aggPaginatedData.Data {
        var v *Video
        if marshallErr := bson.Unmarshal(raw, &v); marshallErr == nil {
            v.SignedURL, _ = storageClient.Bucket(os.Getenv("GOOGLE_CLOUD_STORAGE_BUCKET")).SignedURL(v.UUID, opts)
            results = append(results, *v)
        }

    }

    c.JSON(http.StatusOK, PaginatedResult{Videos: results, Pagination: aggPaginatedData.Pagination})

}
Enter fullscreen mode Exit fullscreen mode

As you can see, Atlas Search provides simple interface to search. All we need is the following line:

    searchStage := bson.M{"$search": bson.D{{"index", os.Getenv("MONGO_DB_SEARCH_INDEX")}, {"text", bson.D{{"path", bson.D{{"wildcard", "*"}}}, {"query", searchBody.Keyword}}}}}
Enter fullscreen mode Exit fullscreen mode

I left the Atlas Search parameters as default, as I discovered that they work great for my usecase. They search across all of the saved video features:

search settings

After that, the video upload process is complete. If you would like to see the whole method, and not only the code snippets, you can access it on the GitHub: https://github.com/tavsec/devto-mongodb-hackathon-api/blob/main/Controllers/VideoController.go

logic flow

Link to Source Code

API

GitHub logo tavsec / devto-mongodb-hackathon-api

API server for Dev.to MongoDB hackathon, using Google Cloud services and Atlas search

devto-mongodb-hackathon-api

API for my DevTo MongoDB hackathon project. https://dev.to/timotej_avsec/video-content-search-using-mongodb-atlas-search-and-google-machine-learning-1f32

Build

Firstly, add you application_default_credentials.json to the source of the project. This will enable Google Cloud API.

Then, edit .env file

mv .env.example .env
Enter fullscreen mode Exit fullscreen mode

Edit contents of the .env file.

Finally, you can run the application as you would any other Golang application:

go build -o devto-mongodb-hackathon github.com/tavsec/devto-mongodb-hackathon-api
Enter fullscreen mode Exit fullscreen mode

You can also use Docker:

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

Frontend

GitHub logo tavsec / devto-mongodb-hackathon-app

Frontend application for DevTo Mongodb hackathon

devto-mongodb-hackathon-app

Frontend application for my DevTo MongoDB hackathon project. https://dev.to/timotej_avsec/video-content-search-using-mongodb-atlas-search-and-google-machine-learning-1f32

Build Setup

# create and update .env file
$ cp .env.example .env

# install dependencies
$ npm install

# serve with hot reload at localhost:3000
$ npm run dev

# build for production and launch server
$ npm run build
$ npm run start

# generate static project
$ npm run generate
Enter fullscreen mode Exit fullscreen mode



Permissive License

Both repositories are licensed as MIT.

Final thoughts

I really enjoyed building this application, as I discovered and learned a lot about MongoDB, as well as Google Cloud APIs. I was also fascinated about ease of use of Atlas Search, as it basically worked out-of-the-box (well, for my use-case, anyways). I will deffinietly use MongoDB for some of my personal projects in the future.

Oldest comments (0)