DEV Community

Keisuke Sato
Keisuke Sato

Posted on • Updated on

Using MLflow on google colaboratory with github to build cosy environment: design

(Updated on 19, March 2022)
(Updated on 6, February 2022)
(Updated on 28, January 2022)

What do I want to do?

It's so troublesome to manage all settings (e.g., epochs, optimizer, etc) for building a certain model and I'd be interested in Kaggle these days. Before I join the competitions, I build a cosy environment.

Pieces of my cosy environment

I'll use the following one library and four services.

  • MLflow Tracking
  • Google Colaboratory
  • Google Drive
  • Github
  • ngrok

MLflow and google drive are for managing settings for building models, google colaboratory is for building models, github is for managing the source codes, and the result of mlflow is seen through ngrok.

Concept

Image description

Apparently, MLflow Projects can connect to gitlab and manage the source codes, but at first, I just directly clone a repository and push new changes from google colaboratory notebook. Because of that, I have to add the commit number information to MLflow information after pushing new source codes on a remote repository.

Discussion (0)