DEV Community

Cover image for Quickstart to Flyte
Hridya for Learn Earn & Fun

Posted on

Quickstart to Flyte

Image description


What is Flyte?

  • Kubernetes-native workflow automation platform
  • Open-source
  • Makes it easy create concurrent, scalable, and maintainable workflows
  • DLF AI & Data Incubation Project
  • Opinionated, scalable & hosted workflow automating platform
  • Extensible, Auditable, Observable


Flyte supports a ton of integrations such as TensorFlow, Google Cloud, Apache Spark, PyTorch, Hive, Kubernetes, etc...

Image description

This is an overall view of how many integrations they support!

You can check out all the integrations they support by clicking here

Trust by Companies

Flyte is used in production at Lyft, Spotify, Freenome and others.

Image description

Setting Up Flyte


  • Docker
  • Python

Ensure that your Docker Daemon is running


Installing Flytekit

pip install flytekit
Enter fullscreen mode Exit fullscreen mode

NOTE: For Mac M1 Users, run this command before running the above command to install grpcio in your computer

pip install --no-binary :all: grpcio --ignore-installed
Enter fullscreen mode Exit fullscreen mode

Installing FlyteCTL

FlyteCTL is a command-line interface for Flyte

brew install flyteorg/homebrew-tap/flytectl
Enter fullscreen mode Exit fullscreen mode
Other Operating Systems
curl -L | bash
Enter fullscreen mode Exit fullscreen mode

Creating an Example Flyte Script

Just to checkout your setup works and have a bit of fun with Flyte.

Let's create an example script with flyte that:

  • Generate a dataset of numbers drawn from a normal distribution.
  • Compute the mean and standard deviation of the numbers data.

Here's the script, insert it into any python file

import typing
import pandas as pd
import numpy as np

from flytekit import task, workflow

def generate_normal_df(n:int, mean: float, sigma: float) -> pd.DataFrame:
    return pd.DataFrame({"numbers": np.random.normal(mean, sigma,size=n)})

def compute_stats(df: pd.DataFrame) -> typing.Tuple[float, float]:
    return float(df["numbers"].mean()), float(df["numbers"].std())

def wf(n: int = 200, mean: float = 0.0, sigma: float = 1.0) -> typing.Tuple[float, float]:
    return compute_stats(df=generate_normal_df(n=n, mean=mean, sigma=sigma))
Enter fullscreen mode Exit fullscreen mode

Running Flyte workflows

You can run the workflow in on a local Python environment or a Flyte cluster.

Running a workflow using a local python env

Run this command to kickstart your newly created workflow using a python env

NOTE: Change with the filename your Python file is!

pyflyte run wf --n 500 --mean 42 --sigma 2
Enter fullscreen mode Exit fullscreen mode

Creating a Demo Flyte Cluster

Run this command to kickstart your newly created workflow using a Flyte Cluster.

flytectl demo start
Enter fullscreen mode Exit fullscreen mode

If you have setup everything correctly, You should recieve the following message:

Image description

Now run the workflow on the cluster using this command:

pyflyte run --remote wf --n 500 --mean 42 --sigma 2
Enter fullscreen mode Exit fullscreen mode

Great! You have run and successfully setup Flyte in your computer


πŸŽ‰ Congratulations! In this getting started guide, you:

  • πŸ€“ You learned all about Flyte
  • πŸ’» Setup Flyte in your computer
  • πŸ“œ Created a Flyte script
  • πŸ›₯ Created a demo Flyte cluster on your local system.
  • πŸ‘Ÿ Ran a workflow locally and on a demo Flyte cluster.

Flyte is a great workflow automation tool for Data, Machine Learning Processes

Lastly, don't forget to leave a LIKE and key in your feedback in the comments!

Top comments (0)

An Animated Guide to Node.js Event Loop

Node.js doesn’t stop from running other operations because of Libuv, a C++ library responsible for the event loop and asynchronously handling tasks such as network requests, DNS resolution, file system operations, data encryption, etc.

What happens under the hood when Node.js works on tasks such as database queries? We will explore it by following this piece of code step by step.