DEV Community

Cover image for Microservice in Python using FastAPI
Paurakh Sharma Humagain
Paurakh Sharma Humagain

Posted on • Updated on

Microservice in Python using FastAPI

Creating Microservices with Python

As a Python developer you might have heard about the term microservices, and want to build a Python microservice by yourself. Microservices is a great architecture for building highly scalable applications. Before starting to build your application using the microservice, you must be familiar with the benefits and drawbacks of using microservices. In this article, you will learn the benefits and drawbacks of using microservices. You will also learn how you can build your own microservice and deploy it using the Docker Compose.

In this tutorial you'll learn:

  • What the benefits and drawbacks of microservices are
  • Why you should build microservice with Python
  • How to build REST API using FastAPI and PostgreSQL
  • How to build microservice using FastAPI
  • How to run microservices using docker-compose
  • How to manage microservices using Nginx

You will first build a simple REST API using FastAPI and then use PostgreSQL as our database. You will then extend the same application to a microservice.

Introduction to Microservices

Microservice is the approach of breaking down large monolith application into individual applications specializing in a specific service/functionality. This approach is often known as Service-Oriented Architecture or SOA.

In monolithic architecture, every business logic resides in the same application. Application services such as user management, authentication, and other features use the same database.

In a microservice architecture, the application is broken down into several separate services that run in separate processes. There is a different database for different functionality of the application and the services communicate with each other using the HTTP, AMQP, or a binary protocol like TCP, depending on the nature of each service. Inter-service communication can also be performed using the message queues like RabbitMQ, Kafka or Redis.

Benefits of Microservice

The microservice architecture comes with lots of benefits. Some of these benefits are:

  • Loosely coupled application means the different services can be build using the technologies that suit them best. So, the development team is not bounded to the choices made while starting the project.

  • Since the services are responsible for specific functionality which makes it easier to understand and keep the application under control.

  • Application scaling also becomes easier because if one of the services requires high GPU usage then only the server consisting that service needs to have high GPU and others can run on a normal server.

Drawbacks of Microservice

The microservice architecture is not a silver bullet that solves all your problems, it comes with its drawbacks too. Some of these drawbacks are:

  • Since different services use the different database the transactions involving more than one service needs to use eventual consistency.

  • Perfect splitting of the services is very difficult to achieve at the first try and this needs to be iterated before coming with the best possible separation of the services.

  • Since the services communicate with each other through the use of network interaction, this makes the application slower due to the network latency and slow service.

Why Microservice in Python

Python is a perfect tool for building micro-services because it comes with a great community, easy learning curve and tons of libraries. Due to the introduction of asynchronous programming in Python, web frameworks with performance on-par with GO and Node.js, has emerged.

Introduction to FastAPI

FastAPI is a modern, high-performance, web framework, which comes with tons of cool features like auto-documentation based on OpenAPI and built-in serialization and validation library. See here for the list of all cool features in FastAPI.

Why FastAPI

Some of the reason why I think FastAPI is a great choice for building microservices in Python is:

  • Auto documentation
  • Async/Await support
  • Built-in validation and serialization
  • 100% type annotated so autocompletion works great

Installing FastAPI

Before installing FastAPI create a new directory movie_service and create a new virtual environment inside the newly created directory using virtualenv.
If you haven't already installed virtualenv:

pip install virtualenv
Enter fullscreen mode Exit fullscreen mode

Now, create a new virtual environment.

virtualenv env
Enter fullscreen mode Exit fullscreen mode

If you are on Mac/Linux you can activate the virtual environment using the command:

source ./env/bin/activate
Enter fullscreen mode Exit fullscreen mode

Windows users can run this command instead:

.\env\Scripts\activate
Enter fullscreen mode Exit fullscreen mode

Finally, Your are ready to install FastAPI, run the following command:

pip install fastapi
Enter fullscreen mode Exit fullscreen mode

Since FastAPI doesn't come with inbuilt service, you need to install uvicorn for it to run. uvicorn is an ASGI server which allows us to use async/await features.
Install uvicorn using the command

pip install uvicorn
Enter fullscreen mode Exit fullscreen mode

Creating Simple REST API using FastAPI

Before You start building a microservice using FastAPI, let's learn the basics of FastAPI. Create a new directory app and a new file main.py inside the newly-created directory.

Add the following code in main.py.

#~/movie_service/app/main.py

from fastapi import FastAPI

app = FastAPI()


@app.get('/')
async def index():
    return {"Real": "Python"}
Enter fullscreen mode Exit fullscreen mode

Here you first import and instantiate the FastAPI and then register the root endpoint / which then returns a JSON.

You can run the application server using uvicorn app.main:app --reload. Here app.main indicates you use main.py file inside the app directory and :app indicates our FastAPI instance name.

You can access the app from http://127.0.0.1:8000. To access the cool automatic documentation, head over to http://127.0.0.1:8000/docs. You can play around and interact with your API from the browser itself.

Let's add some CRUD functionality to our application.
Update your main.py to look like the following:

#~/movie_service/app/main.py

from fastapi import FastAPI
from pydantic import BaseModel
from typing import List

app = FastAPI()

fake_movie_db = [
    {
        'name': 'Star Wars: Episode IX - The Rise of Skywalker',
        'plot': 'The surviving members of the resistance face the First Order once again.',
        'genres': ['Action', 'Adventure', 'Fantasy'],
        'casts': ['Daisy Ridley', 'Adam Driver']
    }
]

class Movie(BaseModel):
    name: str
    plot: str
    genres: List[str]
    casts: List[str]


@app.get('/', response_model=List[Movie])
async def index():
    return fake_movie_db
Enter fullscreen mode Exit fullscreen mode

As you can see you have created a new class Movie which extends BaseModel from pydantic.
The Movie model contains the name, phot, genres, and casts. Pydantic comes built-in with FastAPI which makes making models and request validation a breeze.

If you head over to the docs site you can see there are fields of our Movies model mentioned already in the example response section. This is possible because you have defined the response_model in our route definition.

Now, let's add the endpoint to add a movie to our movies list.

Add a new endpoint definition to handle the POST request.

@app.post('/', status_code=201)
async def add_movie(payload: Movie):
    movie = payload.dict()
    fake_movie_db.append(movie)
    return {'id': len(fake_movie_db) - 1}
Enter fullscreen mode Exit fullscreen mode

Now, head over to the browser and test the new API. Try adding a movie with an invalid field or without the required fields and see that the validation is automatically handled by FastAPI.

Let's add a new endpoint to update the movie.

@app.put('/{id}')
async def update_movie(id: int, payload: Movie):
    movie = payload.dict()
    movies_length = len(fake_movie_db)
    if 0 <= id <= movies_length:
        fake_movie_db[id] = movie
        return None
    raise HTTPException(status_code=404, detail="Movie with given id not found")
Enter fullscreen mode Exit fullscreen mode

Here id is the index of our fake_movie_db list.

Note: Remember to import HTTPException from fastapi

Now you can also add the endpoint to delete the movie.

@app.delete('/{id}')
async def delete_movie(id: int):
    movies_length = len(fake_movie_db)
    if 0 <= id <= movies_length:
        del fake_movie_db[id]
        return None
    raise HTTPException(status_code=404, detail="Movie with given id not found")
Enter fullscreen mode Exit fullscreen mode

Before you move forward, let's structure our app in a better way. Create a new folder api inside app and create a new file movies.py inside the recently created folder. Move all the routes related codes from main.py to movies.py. So, the movies.py should look like the following:

#~/movie-service/app/api/movies.py

from typing import List
from fastapi import Header, APIRouter

from app.api.models import Movie

fake_movie_db = [
    {
        'name': 'Star Wars: Episode IX - The Rise of Skywalker',
        'plot': 'The surviving members of the resistance face the First Order once again.',
        'genres': ['Action', 'Adventure', 'Fantasy'],
        'casts': ['Daisy Ridley', 'Adam Driver']
    }
]

movies = APIRouter()

@movies.get('/', response_model=List[Movie])
async def index():
    return fake_movie_db

@movies.post('/', status_code=201)
async def add_movie(payload: Movie):
    movie = payload.dict()
    fake_movie_db.append(movie)
    return {'id': len(fake_movie_db) - 1}

@movies.put('/{id}')
async def update_movie(id: int, payload: Movie):
    movie = payload.dict()
    movies_length = len(fake_movie_db)
    if 0 <= id <= movies_length:
        fake_movie_db[id] = movie
        return None
    raise HTTPException(status_code=404, detail="Movie with given id not found")

@movies.delete('/{id}')
async def delete_movie(id: int):
    movies_length = len(fake_movie_db)
    if 0 <= id <= movies_length:
        del fake_movie_db[id]
        return None
    raise HTTPException(status_code=404, detail="Movie with given id not found")
Enter fullscreen mode Exit fullscreen mode

Here you registered a new API route using the APIRouter from FastAPI.

Also, create a new file models.py inside api where you will be keeping our Pydantic models.

#~/movie-service/api/models.py

from typing import List
from pydantic import BaseModel

class Movie(BaseModel):
    name: str
    plot: str
    genres: List[str]
    casts: List[str]

Enter fullscreen mode Exit fullscreen mode

Now register this new routes file in main.py

#~/movie-service/app/main.py
from fastapi import FastAPI

from app.api.movies import movies

app = FastAPI()

app.include_router(movies)
Enter fullscreen mode Exit fullscreen mode

Finally, our application directory structure looks like this:

movie-service
├── app
│   ├── api
│   │   ├── models.py
│   │   ├── movies.py
│   |── main.py
└── env
Enter fullscreen mode Exit fullscreen mode

Make sure your application is working properly before you move forward.

Using PostgreSQL Database with FastAPI

Previously, you used fake Python list to add movies but now you are finally ready to use an actual database for this purpose. You are going to use PostgreSQL for this purpose. Install PostgreSQL if you haven't already. After installing the PostgreSQl create a new database, I am going to call mine movie_db.

You are going to use encode/databases to connect to the database using async and await support. Learn more about async/await in Python here

Install the required library using:

pip install 'databases[postgresql]'
Enter fullscreen mode Exit fullscreen mode

this will install sqlalchemy and asyncpg as well, which are required for working with PostgreSQL.

Create a new file inside api and call it db.py. This file will contain the actual database model for our REST API.

#~/movie-service/app/api/db.py

from sqlalchemy import (Column, Integer, MetaData, String, Table,
                        create_engine, ARRAY)

from databases import Database

DATABASE_URL = 'postgresql://movie_user:movie_password@localhost/movie_db'

engine = create_engine(DATABASE_URL)
metadata = MetaData()

movies = Table(
    'movies',
    metadata,
    Column('id', Integer, primary_key=True),
    Column('name', String(50)),
    Column('plot', String(250)),
    Column('genres', ARRAY(String)),
    Column('casts', ARRAY(String))
)

database = Database(DATABASE_URL)
Enter fullscreen mode Exit fullscreen mode

Here, DATABASE_URI is the URL used to connect to the PostgreSQL database. Here movie_user is the name of the database user, movie_password is the password of the database user and movie_db is the name of the database.

Just like you would to in SQLAlchemy you have created the table for the movies database.

Update main.py to connect to the database. main.py should look like the following:

#~/movie-service/app/main.py

from fastapi import FastAPI
from app.api.movies import movies
from app.api.db import metadata, database, engine

metadata.create_all(engine)

app = FastAPI()

@app.on_event("startup")
async def startup():
    await database.connect()

@app.on_event("shutdown")
async def shutdown():
    await database.disconnect()


app.include_router(movies)
Enter fullscreen mode Exit fullscreen mode

FastAPI provides some event handlers which you can use to connect to our database when the application starts and disconnect when it shuts down.

Update movies.py so that it uses a database instead of a fake Python list.

#~/movie-service/app/api/movies.py


from typing import List
from fastapi import Header, APIRouter

from app.api.models import MovieIn, MovieOut
from app.api import db_manager

movies = APIRouter()

@movies.get('/', response_model=List[MovieOut])
async def index():
    return await db_manager.get_all_movies()

@movies.post('/', status_code=201)
async def add_movie(payload: MovieIn):
    movie_id = await db_manager.add_movie(payload)
    response = {
        'id': movie_id,
        **payload.dict()
    }

    return response

@movies.put('/{id}')
async def update_movie(id: int, payload: MovieIn):
    movie = payload.dict()
    fake_movie_db[id] = movie
    return None

@movies.put('/{id}')
async def update_movie(id: int, payload: MovieIn):
    movie = await db_manager.get_movie(id)
    if not movie:
        raise HTTPException(status_code=404, detail="Movie not found")

    update_data = payload.dict(exclude_unset=True)
    movie_in_db = MovieIn(**movie)

    updated_movie = movie_in_db.copy(update=update_data)

    return await db_manager.update_movie(id, updated_movie)

@movies.delete('/{id}')
async def delete_movie(id: int):
    movie = await db_manager.get_movie(id)
    if not movie:
        raise HTTPException(status_code=404, detail="Movie not found")
    return await db_manager.delete_movie(id)
Enter fullscreen mode Exit fullscreen mode

Let's add db_manager.py to manipulate our database.

#~/movie-service/app/api/db_manager.py

from app.api.models import MovieIn, MovieOut, MovieUpdate
from app.api.db import movies, database


async def add_movie(payload: MovieIn):
    query = movies.insert().values(**payload.dict())

    return await database.execute(query=query)

async def get_all_movies():
    query = movies.select()
    return await database.fetch_all(query=query)

async def get_movie(id):
    query = movies.select(movies.c.id==id)
    return await database.fetch_one(query=query)

async def delete_movie(id: int):
    query = movies.delete().where(movies.c.id==id)
    return await database.execute(query=query)

async def update_movie(id: int, payload: MovieIn):
    query = (
        movies
        .update()
        .where(movies.c.id == id)
        .values(**payload.dict())
    )
    return await database.execute(query=query)
Enter fullscreen mode Exit fullscreen mode

Let's update our models.py so that you can use the Pydantic model with the sqlalchemy table.

#~/movie-service/app/api/models.py

from pydantic import BaseModel
from typing import List, Optional

class MovieIn(BaseModel):
    name: str
    plot: str
    genres: List[str]
    casts: List[str]


class MovieOut(MovieIn):
    id: int


class MovieUpdate(MovieIn):
    name: Optional[str] = None
    plot: Optional[str] = None
    genres: Optional[List[str]] = None
    casts: Optional[List[str]] = None
Enter fullscreen mode Exit fullscreen mode

Here MovieIn is the base model that you use to add the movie to the database. You have to add the id to this model while getting it from the database, hence the MovieOut model. MovieUpdate model allows us to set the values in the model to be optional so that while updating the movie only the field that needs to be updated can be sent.

Now, head over to the browser documentation site and start playing with the API.

Microservice Data Management Patterns

Managing data in microservice is one of the most challenging aspects of building a microservice. Since different functions of the application are handled by different services, usage of a database can be tricky.

Here are some patterns that you can use to manage data flow in the application.

Database Per Service

Using a database per service is great if you want your microservices to be as loosely coupled as possible. Having a different database per service allows us to scale different services independently. A transaction involving multiple databases is done through well-defined APIs. This comes with its drawback as implementing business transactions involving multiple services is not straightforward. Also, the addition of network overhead makes this less efficient to use.

Shared Database

If there are lots of transactions involving multiple services it is better to use a shared database. This comes with the benefits of highly consistent application but takes away most of the benefits that come with the microservices architecture. Developers working on one service needs to coordinate with the schema changes in other services.

API Composition

In transactions involving multiple databases, API composer acts as an API gateway and executes API calls to other microservices in the required order. Finally, the results from each microservices are returned to the client service after performing an in-memory join. The downside of this approach is inefficient in-memory joins of a large dataset.

Creating a Python Microservice in Docker

The pain of deploying the microservice can be greatly reduced by using the Docker. Docker helps to encapsulate each service and scale them independently.

Installing Docker and Docker Compose

If you haven't already install docker in your system. Verify if the docker is installed by running the command docker. After you have done installing Docker, install Docker Compose. Docker Compose is used for defining and running multiple Docker containers. It also helps in easy interaction between them.

Creating Movies Service

Since a lot of the work for building a movie service is already done while getting started with the FastAPI, you are going to reuse the code you have already written. Create a brand new folder, I am going to call mine python-microservices. Move the code you wrote earlier which I had named movie-service.
So, the folder structure would look like this:

python-microservices/
└── movie-service/
    ├── app/
    └── env/
Enter fullscreen mode Exit fullscreen mode

First of all, let's create a requirements.txt file where you are going to keep all the dependencies you are going to use in our movie-service.
Create a new file requirements.txt inside movie-service and add the following to it:

asyncpg==0.20.1
databases[postgresql]==0.2.6
fastapi==0.48.0
SQLAlchemy==1.3.13
uvicorn==0.11.2
httpx==0.11.1
Enter fullscreen mode Exit fullscreen mode

You have used all the libraries mentioned there except httpx which you are going to use while making service to service API call.

Create a Dockerfile inside movie-service with the following contents:

FROM python:3.8-slim

WORKDIR /app

COPY ./requirements.txt /app/requirements.txt

RUN apt-get update \
    && apt-get install gcc -y \
    && apt-get clean

RUN pip install -r /app/requirements.txt \
    && rm -rf /root/.cache/pip

COPY . /app/
Enter fullscreen mode Exit fullscreen mode

Here first, you define which Python version you want to use. Then set the WORKDIR to be inside app folder inside the Docker container. After that gcc is installed which is required by the libraries that you are using in the application.
Finally, install all dependencies in requirements.txt and copy all the files inside movie-service/app.

Update db.py and replace

DATABASE_URI = 'postgresql://movie_user:movie_password@localhost/movie_db'
Enter fullscreen mode Exit fullscreen mode

with

DATABASE_URI = os.getenv('DATABASE_URI')
Enter fullscreen mode Exit fullscreen mode

Note: Don't forget to import os on the top of the file.

You need to do this so that you can latter provide DATABASE_URI as an environment variable.

Also, update main.py and replace

app.include_router(movies)
Enter fullscreen mode Exit fullscreen mode

with

app.include_router(movies, prefix='/api/v1/movies', tags=['movies'])
Enter fullscreen mode Exit fullscreen mode

Here, you have added prefix /api/v1/movies so, that managing different version of API becomes easier. Also, tags make finding API related to movies easier in FastAPI docs.

Also, you need to update our models so that the casts stores the cast's id instead of the actual name. So, update the models.py to look like this:

#~/python-microservices/movie-service/app/api/models.py

from pydantic import BaseModel
from typing import List, Optional

class MovieIn(BaseModel):
    name: str
    plot: str
    genres: List[str]
    casts_id: List[int]


class MovieOut(MovieIn):
    id: int


class MovieUpdate(MovieIn):
    name: Optional[str] = None
    plot: Optional[str] = None
    genres: Optional[List[str]] = None
    casts_id: Optional[List[int]] = None
Enter fullscreen mode Exit fullscreen mode

Likewise, you need to update the database tables, let's update db.py:

#~/python-microservices/movie-service/app/api/db.py

import os

from sqlalchemy import (Column, DateTime, Integer, MetaData, String, Table,
                        create_engine, ARRAY)

from databases import Database

DATABASE_URL = os.getenv('DATABASE_URL')

engine = create_engine(DATABASE_URL)
metadata = MetaData()

movies = Table(
    'movies',
    metadata,
    Column('id', Integer, primary_key=True),
    Column('name', String(50)),
    Column('plot', String(250)),
    Column('genres', ARRAY(String)),
    Column('casts_id', ARRAY(Integer))
)

database = Database(DATABASE_URL)
Enter fullscreen mode Exit fullscreen mode

Now, update movies.py to check if the cast with the given id presents in cast service before adding a new movie or updating a movie.

#~/python-microservices/movie-service/app/api/movies.py

from typing import List
from fastapi import APIRouter, HTTPException

from app.api.models import MovieOut, MovieIn, MovieUpdate
from app.api import db_manager
from app.api.service import is_cast_present

movies = APIRouter()

@movies.post('/', response_model=MovieOut, status_code=201)
async def create_movie(payload: MovieIn):
    for cast_id in payload.casts_id:
        if not is_cast_present(cast_id):
            raise HTTPException(status_code=404, detail=f"Cast with id:{cast_id} not found")

    movie_id = await db_manager.add_movie(payload)
    response = {
        'id': movie_id,
        **payload.dict()
    }

    return response

@movies.get('/', response_model=List[MovieOut])
async def get_movies():
    return await db_manager.get_all_movies()

@movies.get('/{id}/', response_model=MovieOut)
async def get_movie(id: int):
    movie = await db_manager.get_movie(id)
    if not movie:
        raise HTTPException(status_code=404, detail="Movie not found")
    return movie

@movies.put('/{id}/', response_model=MovieOut)
async def update_movie(id: int, payload: MovieUpdate):
    movie = await db_manager.get_movie(id)
    if not movie:
        raise HTTPException(status_code=404, detail="Movie not found")

    update_data = payload.dict(exclude_unset=True)

    if 'casts_id' in update_data:
        for cast_id in payload.casts_id:
            if not is_cast_present(cast_id):
                raise HTTPException(status_code=404, detail=f"Cast with given id:{cast_id} not found")

    movie_in_db = MovieIn(**movie)

    updated_movie = movie_in_db.copy(update=update_data)

    return await db_manager.update_movie(id, updated_movie)

@movies.delete('/{id}', response_model=None)
async def delete_movie(id: int):
    movie = await db_manager.get_movie(id)
    if not movie:
        raise HTTPException(status_code=404, detail="Movie not found")
    return await db_manager.delete_movie(id)
Enter fullscreen mode Exit fullscreen mode

Let's add a service to make an API call to cast service:

#~/python-microservices/movie-service/app/api/service.py

import os
import httpx

CAST_SERVICE_HOST_URL = 'http://localhost:8002/api/v1/casts/'
url = os.environ.get('CAST_SERVICE_HOST_URL') or CAST_SERVICE_HOST_URL

def is_cast_present(cast_id: int):
    r = httpx.get(f'{url}{cast_id}')
    return True if r.status_code == 200 else False

Enter fullscreen mode Exit fullscreen mode

You make an api call to get the cast with the given id and return true if the cast exists and false otherwise.

Creating Casts Service

Similar to a movie-service, for creating a casts-service you are going to use FastAPI and PostgreSQL database.

Create a folder structure like the following:

python-microservices/
.
├── cast_service/
│   ├── app/
│   │   ├── api/
│   │   │   ├── casts.py
│   │   │   ├── db_manager.py
│   │   │   ├── db.py
│   │   │   ├── models.py
│   │   ├── main.py
│   ├── Dockerfile
│   └── requirements.txt
├── movie_service/
...
Enter fullscreen mode Exit fullscreen mode

Add the following to requirements.txt:

asyncpg==0.20.1
databases[postgresql]==0.2.6
fastapi==0.48.0
SQLAlchemy==1.3.13
uvicorn==0.11.2
Enter fullscreen mode Exit fullscreen mode

Dockerfile:

FROM python:3.8-slim

WORKDIR /app

COPY ./requirements.txt /app/requirements.txt

RUN apt-get update \
    && apt-get install gcc -y \
    && apt-get clean

RUN pip install -r /app/requirements.txt \
    && rm -rf /root/.cache/pip

COPY . /app/
Enter fullscreen mode Exit fullscreen mode

main.py

#~/python-microservices/cast-service/app/main.py

from fastapi import FastAPI
from app.api.casts import casts
from app.api.db import metadata, database, engine

metadata.create_all(engine)

app = FastAPI()

@app.on_event("startup")
async def startup():
    await database.connect()

@app.on_event("shutdown")
async def shutdown():
    await database.disconnect()

app.include_router(casts, prefix='/api/v1/casts', tags=['casts'])
Enter fullscreen mode Exit fullscreen mode

You have added the prefix of /api/v1/casts so that managing the API becomes easier. Also, adding tags makes finding docs related to casts in the FastAPI docs easier.

casts.py

#~/python-microservices/cast-service/app/api/casts.py

from fastapi import APIRouter, HTTPException
from typing import List

from app.api.models import CastOut, CastIn, CastUpdate
from app.api import db_manager

casts = APIRouter()

@casts.post('/', response_model=CastOut, status_code=201)
async def create_cast(payload: CastIn):
    cast_id = await db_manager.add_cast(payload)

    response = {
        'id': cast_id,
        **payload.dict()
    }

    return response

@casts.get('/{id}/', response_model=CastOut)
async def get_cast(id: int):
    cast = await db_manager.get_cast(id)
    if not cast:
        raise HTTPException(status_code=404, detail="Cast not found")
    return cast
Enter fullscreen mode Exit fullscreen mode

db_manager.py

#~/python-microservices/cast-service/app/api/db_manager.py

from app.api.models import CastIn, CastOut, CastUpdate
from app.api.db import casts, database


async def add_cast(payload: CastIn):
    query = casts.insert().values(**payload.dict())

    return await database.execute(query=query)

async def get_cast(id):
    query = casts.select(casts.c.id==id)
    return await database.fetch_one(query=query)
Enter fullscreen mode Exit fullscreen mode

db.py

#~/python-microservices/cast-service/app/api/db.py

import os

from sqlalchemy import (Column, Integer, MetaData, String, Table,
                        create_engine, ARRAY)

from databases import Database

DATABASE_URI = os.getenv('DATABASE_URI')

engine = create_engine(DATABASE_URI)
metadata = MetaData()

casts = Table(
    'casts',
    metadata,
    Column('id', Integer, primary_key=True),
    Column('name', String(50)),
    Column('nationality', String(20)),
)

database = Database(DATABASE_URI)
Enter fullscreen mode Exit fullscreen mode

models.py

#~/python-microservices/cast-service/app/api/models.py

from pydantic import BaseModel
from typing import List, Optional

class CastIn(BaseModel):
    name: str
    nationality: Optional[str] = None


class CastOut(CastIn):
    id: int


class CastUpdate(CastIn):
    name: Optional[str] = None
Enter fullscreen mode Exit fullscreen mode

Running the microservice using Docker Compose

To run the microservices, create a docker-compose.yml file and add the following to it:

version: '3.7'

services:
  movie_service:
    build: ./movie-service
    command: uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
    volumes:
      - ./movie-service/:/app/
    ports:
      - 8001:8000
    environment:
      - DATABASE_URI=postgresql://movie_db_username:movie_db_password@movie_db/movie_db_dev
      - CAST_SERVICE_HOST_URL=http://cast_service:8000/api/v1/casts/

  movie_db:
    image: postgres:12.1-alpine
    volumes:
      - postgres_data_movie:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=movie_db_username
      - POSTGRES_PASSWORD=movie_db_password
      - POSTGRES_DB=movie_db_dev

  cast_service:
    build: ./cast-service
    command: uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
    volumes:
      - ./cast-service/:/app/
    ports:
      - 8002:8000
    environment:
      - DATABASE_URI=postgresql://cast_db_username:cast_db_password@cast_db/cast_db_dev

  cast_db:
    image: postgres:12.1-alpine
    volumes:
      - postgres_data_cast:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=cast_db_username
      - POSTGRES_PASSWORD=cast_db_password
      - POSTGRES_DB=cast_db_dev

volumes:
  postgres_data_movie:
  postgres_data_cast:
Enter fullscreen mode Exit fullscreen mode

Here you have 4 different services, movie_service, a database for movie_service, cast_service and a database for cast service. You have exposed movie_service to port 8001 similarly cast_service to port 8002.

For the database, you have used volumes so that the data doesn't get destroyed when the docker container is shutdown.

Run the docker-compose using the command:

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

This creates the docker image if it doesn't already exist and runs them.

Head over to http://localhost:8002/docs to add a cast in casts service. Similarly, http://localhost:8001/docs to add the movie in the movie service.

Using Nginx to Access Both Services using a Single Host Address

You have deployed the microservices using Docker compose but there is one minor problem. Each of the microservices needs to be accessed using a different Port. You can solve this issue using Nginx reverse proxy, using Nginx you can direct the request add a middleware which routes our requests to different services based on the API URL.

Add a new file nginx_config.conf inside python-microservices with the following contents.


server {
  listen 8080;

  location /api/v1/movies {
    proxy_pass http://movie_service:8000/api/v1/movies;
  }

  location /api/v1/casts {
    proxy_pass http://cast_service:8000/api/v1/casts;
  }

}
Enter fullscreen mode Exit fullscreen mode

Here you are running the Nginx at port 8080 and routing the requests to movie service if the endpoint starts with /api/v1/movies and similarly to cast service if the endpoint starts with /api/v1/casts

Now, you need to add the nginx service in our docker-compose-yml. Add the following service after cast_db service:

nginx:
    image: nginx:latest
    ports:
      - "8080:8080"
    volumes:
      - ./nginx_config.conf:/etc/nginx/conf.d/default.conf
    depends_on:
      - cast_service
      - movie_service
Enter fullscreen mode Exit fullscreen mode

Now, shut down the containers with the command:

docker-compose down
Enter fullscreen mode Exit fullscreen mode

And run it back again with:

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

Now, you can access both movie service and cast service at port 8080.
Head over to http://localhost:8080/api/v1/movies/ to get the list of movies.

Now, you might be wondering how you can access the docs of the services. For that update main.py of movie service and replace

app = FastAPI()
Enter fullscreen mode Exit fullscreen mode

with

app = FastAPI(openapi_url="/api/v1/movies/openapi.json", docs_url="/api/v1/movies/docs")
Enter fullscreen mode Exit fullscreen mode

Similarly, for cast service replace it with

app = FastAPI(openapi_url="/api/v1/casts/openapi.json", docs_url="/api/v1/casts/docs")
Enter fullscreen mode Exit fullscreen mode

Here, you changed which endpoint the docs are served and from where the openapi.json is served.

Now, you can access the docs from http://localhost:8080/api/v1/movies/docs for movie service and from http://localhost:8080/api/v1/casts/docs for casts service.

If you are stuck at some point or just want to have a look at the complete code, head over to the Github Repo

Conclusion and Next Step

The microservice architecture is great for breaking down a large monolith application into separate business logics but this comes with the complication too. Python is great for building microservice because of the developer experience and tons of packages and frameworks to make developers more productive.

Deploying microservices has become easier thanks to Docker. Learn more on How to develop microservices using Docker, and Docker Compose

Want me to cover any topic? Let me know at twitter or write a comment down below.

Oldest comments (44)

Collapse
 
minchulkim87 profile image
Min

Wow. Thank you so much! I was looking for something like this.

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

Awesome 😊
Let me know what you would like to see next.

Collapse
 
sobolevn profile image
Nikita Sobolev

Thanks a lot for your introduction with FastAPI.

Consider using wemake-python-styleguide for your next FastAPI project.

It is the strictest Python linter out there. It will help you to find possible errors in your code early, show you possible refactoring opportunities, and enforce consistency across the project's codebase.

Check it out:

GitHub logo wemake-services / wemake-python-styleguide

The strictest and most opinionated python linter ever!

wemake-python-styleguide

wemake.services Supporters Build Status codecov Python Version wemake-python-styleguide


Welcome to the strictest and most opinionated Python linter ever.

wemake-python-styleguide logo

wemake-python-styleguide is actually a flake8 plugin with some other plugins as dependencies.

Quickstart

pip install wemake-python-styleguide
Enter fullscreen mode Exit fullscreen mode

You will also need to create a setup.cfg file with the configuration.

Try it online!

We highly recommend to also use:

  • flakeheaven for easy integration into a legacy codebase
  • nitpick for sharing and validating configuration across multiple projects

Running

flake8 your_module.py
Enter fullscreen mode Exit fullscreen mode

This app is still just good old flake8 And it won't change your existing workflow.

invocation results

See "Usage" section in the docs for examples and integrations.

We also support GitHub Actions as first class-citizens Try it out!

Strict is the new cool

Strict linting offers the following benefits to developers and companies:

  1. Ensures consistency - no matter who works on it, the end product will always be the same dependable code
  2. Helps avoid potential bugs - strict rules make…
Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

Sure, I will definitely have a look at it.
Thanks for sharing 🙂

Collapse
 
demianbrecht profile image
Demian Brecht

This is a great, great post that touches many things. FastAPI has been on my back burner for quite some time to try out so thanks for sharing!

A question about async database queries though: Last time I looked, depending on where your database is (same network boundary as your app), you can actually incur a performance penalty by using the event loop. Do you know if that's still the case? I'm imagining a lot of work has gone into that since I last looked so maybe it's not as noticeable. That said, you'd still be freeing up the worker with the async call, so depending on your application needs, taking that performance hit may still be best.

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

I am glad that you liked it.
I am not so sure about it either.

Collapse
 
osema profile image
osema

wooow thank u very much bro great tuto
one thing is missing i need it fastapi oauth2

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

I am glad that you like the post. I will try to come up with an article covering it.

Collapse
 
tadashii1417 profile image
tadashii1417

Thanks a lot for your article, really appreciate that !

But I have a little problem, when I clone the code and run "docker-compose up -d", it return
"502 Bad Gateway"

Do you know how to fix this ? I quite new to docker and nginx :v

Thanks in advance !

Collapse
 
nyancodeid profile image
Ryan Aunur Rassyid • Edited

same here

[SOLVED]
Check your docker-compose logs to see what problem came from.

docker-compose logs

Collapse
 
uwevanopfern profile image
UweVanOpfern • Edited

I was facing the issue, fixed by solving this BUG, because in docker-compose.yml we used DATABASE_URI under movie-service, change it in movie-service\app\api\db.py as well, replace everywhere you have DATABASE_URL to DATABASE_URI

Collapse
 
bagonzalo profile image
Curated

Major bugs:
(1) you need to initialize first de dabases in the "docker-compose.yml". Otherwise the services try to conect (in my case) to them without being initialize.
(2) change DATABASE_URL for DATABASE_URI here:
"""

~/python-microservices/movie-service/app/api/db.py

import os

from sqlalchemy import (Column, DateTime, Integer, MetaData, String, Table,
create_engine, ARRAY)

from databases import Database

DATABASE_URL = os.getenv('DATABASE_URL')
"""

Collapse
 
bagonzalo profile image
Curated

And thanks indeed for the post.

Collapse
 
uwevanopfern profile image
UweVanOpfern

Helped me, a lot, thanks

Collapse
 
olegasdo profile image
olegasdo

Thank you.

Very nice article

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

Thank you 🙂

Collapse
 
lioncoding profile image
Laurent Egbakou • Edited

Very nice article !
Please, can you add Travis or GitHub actions to the project and deploy it on heroku ?
Maybe update the article too. It will be nice !

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

Thanks for the suggestion. I will try to come up with another article regarding deployment.

Collapse
 
mpavlov profile image
Marat Pavlov

Thanks!
1q: why did you use venv instead pipenv? I saw your previous articles and you use pipenv there

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

I think people are more familiar with venv instead fo pipenv so I choose venv.. But you can use pipenv instead, if you like it 🙂

Collapse
 
turningpro profile image
turningpro

Excellent article. Looking forward to read more of your posts.

Collapse
 
turningpro profile image
turningpro

If you can do a variation with mongodb would be much appreciated.
Not much content out there on that stack.

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

Thank you for following the article. I am glad it helped you.

Thank you for the suggestion, I will try to come up with the article for using FastAPI with Momgodb

Thread Thread
 
waldirborbajr profile image
Waldir Borba Jr

I earned another fan. Yes, please make an article FastAPI + Mongo... pls, pls plssssss

Collapse
 
coolguy1210 profile image
Aravinth Ponnusamy

Hey Paurakh, first of all, thanks for the post very helpful. I am trying to understand a few things here any guidance would be appreciated. I am a novice so, pls excuse if my questions are lame,

Q1 - I tried the example that you have given were there were only 4 fields name, plot, genres, casts
everything is fine. Now if I need to make this work for let's say mongo, I need to add 2 more fields id and year, how would you go about doing that?

fake_movie_db = [
{
'name': 'Star Wars: Episode IX - The Rise of Skywalker',
'plot': 'The surviving members of the resistance face the First Order once again.',
'genres': ['Action', 'Adventure', 'Fantasy'],
'casts': ['Daisy Ridley', 'Adam Driver']
}
]

Q2 - when I tried the same with our existing code, POST request was successful but when do a get it was not showing any of the new fields. there should be an error in this scenario?

Thanks
Aravinth

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

No, this is not a lame question. Thank you for asking the question.

To achieve this you have to add id and year to your Movie model.

e.g

class Movie(BaseModel):
    name: str
    plot: str
    genres: List[str]
    casts: List[str]
    id: str
    year: str

I hope that answers your question. Please let me know how that goes.

Collapse
 
plaoo profile image
Paolo Monni

Beautiful article, following what you made I created a personal "library" project. I would like to ask you if you can add the way to do the tests on the various endpoints, because following the fastapi documentation I find myself having problems with the db

Collapse
 
tomarnaez profile image
TomArnaez • Edited

Thanks for this. I wanted to ask what's the easiest way to define a relationship a many-to-many when using encode/database?

Collapse
 
spansky profile image
Leon

An excellent article. Thanks a lot.

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

Thank you so much, I am glad that you liked it 😊

Collapse
 
cimmanuel profile image
Immanuel Kolapo

Hello @paurakhsharma , I tried using these steps to create another project but when I try inserting in the database, I get asyncpg.exceptions.DataError: invalid input for query argument $1: 213127865166 (value out of int32 range). What could be wrong? Kindly assist!
You can see this stack overflow for more https://stackoverflow.com/questions/63404139/asyncpg-exceptions-dataerror-invalid-input-for-query-argument-1-217027642536

Collapse
 
paurakhsharma profile image
Paurakh Sharma Humagain

I see you have already accepted the answer in StackOverflow, good luck :)