DEV Community

Cover image for Ease your development hurdles with these [M]ake recipes
David Jiménez
David Jiménez

Posted on • Updated on

Ease your development hurdles with these [M]ake recipes

Table of contents

  1. Introduction
  2. Some Recipes/Tasks for your backend projects
    1. Build the project
    2. Clean up the workspace
    3. Run the project
    4. Prepare the local environment for development
    5. Reset the database
    6. Freeze and unfreeze the state of the database
    7. Update non-local environments
    8. File generation scripts
    9. Run arbitrary command
    10. Migrate database
    11. Check code
    12. Bump version
    13. Run unit/acceptance tests
    14. Open the profiling tool
  3. Conclusions

Introduction

Make and other similar expert/automation systems like PyInvoke (I highly love this one over other build systems, even over Makefile), Rake, Cake, etc... avoid the necessity of having a bunch of scripts (bash, python...) to perform certain repetitive tasks. Even then, you often chain them in succession to achieve a certain goal (e.g. you know you have to run script_1.sh before script_2.sh, but that dependency is probably not made explicit in the project file structure). These dependencies are difficult to remember as they become more and more complex.

Or you might install a whole bunch of other tools for your project: poetry, gulp or docker just to name a few. Eventually recalling the arguments required to perform a certain action becomes difficult. But it's never difficult to recall the action you need to perform.

Dumb metaphor: It's easier to remember that you need an spanish omelette than remember that you need eggs, potatoes, onions and salt to build such delicious dish. Wouldn't it be easier if you had the power to imagine an omelette and PUFF!, it materializes in front of you, than building the omelette by yourself each and every time? That's where make and other derivatives can help you.

Here I present you with several suggested recipes that I use on a daily basis on the backend projects I have been working. The example are expressed in Makefile syntax, but the very idea can be applied to every build system.

Some Recipes/Tasks for your backend projects

Build the project (make build)

Pretty self explanatory. Every project of a reasonable size and maturity needs to be built, either by compilation, building the docker image, you name it. Therefore, the responsibility of this recipe should be to build/update the project and leave it in a runnable state. Optionally it could run migrations as well to keep the database schema up to date.

Make recipe:

build:
        # Put here the commands to compile, install dependencies, build docker images...
Enter fullscreen mode Exit fullscreen mode

Invocation:

make build
Enter fullscreen mode Exit fullscreen mode

Clean up the workspace (make clean)

This one is the inverse of build. Its responsibility is to clean up the docker images, volumes, generated files... created by the build command.

Make recipe:

clean:
        # Clean docker images, dist files, etc...
Enter fullscreen mode Exit fullscreen mode

Invocation:

make clean
Enter fullscreen mode Exit fullscreen mode

Run the project (make run)

This one runs the project. Prior to spinning up the application (running docker-compose up, executing the main script, etc.), it could prepare the environment to ensure that the project starts up successfully: invalidating caches, creating an AWS session, running database migrations, etc...

This is the command we tell the frontend team to run after building the application with make init to spin up the web server for theirs to run their HTTP requests.

Make recipe:

run:
        # Run your project here

stop:
        # In the same sense that there is a run, it might be interesting for you to have a stop command that kills the process/es that starts up the `run` command.
Enter fullscreen mode Exit fullscreen mode

Invocation:

make run
make stop
Enter fullscreen mode Exit fullscreen mode

Prepare the local environment for development (make setup-dev)

This one is intended for those that are going to do development over the project. It installs dependencies locally, set up pre-commit hooks, etc... basically setup and install anything required for local development.

Make recipe:

setup-dev:
        # Run pre-commit scripts, install dependencies locally...
Enter fullscreen mode Exit fullscreen mode

Invocation:

make setup-dev
Enter fullscreen mode Exit fullscreen mode

Reset the database (make reset)

Eventually and through your testing or the testing of your colleague developers, the database is going to be heavily polluted, and this command should reset the database to factory settings (destroying and creating the local database, re-running the initial data migrations, etc.)

Make recipe:

reset: down
        # Run db scripts to purge the DB/s, remove docker volumes...
Enter fullscreen mode Exit fullscreen mode

Invocation:

make reset
Enter fullscreen mode Exit fullscreen mode

Freeze and unfreeze the state of the database (make freeze and make unfreeze)

I came up with these after some people of my team suggested that it would be cool, from a testing perspective, if we could save a snapshot of the database and then restore it. It proved very useful.

Make recipe:

# Example with postgres, leveraging docker

freeze: down
        docker run -v my_postgres_vol:/volume -v /tmp:/backup --rm loomchild/volume-backup backup postgres

unfreeze: down
    docker run -v my_postgres_vol:/volume -v /tmp:/backup --rm loomchild/volume-backup restore postgres
Enter fullscreen mode Exit fullscreen mode

Invocation:

make freeze
make unfreeze
Enter fullscreen mode Exit fullscreen mode

Update non-local environments like DEV/TEST/UAT/PROD (make update-remote)

Often non-local environments have different characteristics than your machine (they might not run via docker, for example). Having a recipe to handle migrations and updates to those environments is very useful to avoid forgetting to run something.

Make recipe:

update-remote:
        # call here the commands that are intended to be called when you deploy to your remote environment: Database migrations, cache invalidations, scripts to fix something...
Enter fullscreen mode Exit fullscreen mode

Invocation:

make update-remote
Enter fullscreen mode Exit fullscreen mode

File generation scripts (make generate-XXXX)

Sometimes you might need to generate files in your code (documentation, protobuf code, etc...). These kind of commands are intended for such tasks.

Make recipe:

generate-docs:
        # call your doc generation tool (doxygen, sphynx...)
Enter fullscreen mode Exit fullscreen mode

Invocation:

make generate-docs
Enter fullscreen mode Exit fullscreen mode

Run arbitrary command (make run-command)

I work with Django (but this probably applies to other frameworks as well). And wrapping these commands into a recipe is often a good idea to keep the execution centralized.

Make recipe:

run-command:
        # e.g. python src/manage.py ($cmd)
Enter fullscreen mode Exit fullscreen mode

Invocation:

make run-command cmd="XXXXX"
Enter fullscreen mode Exit fullscreen mode

Migrate database (make migrate)

This recipe usually is a dependency of another of the previously mentioned recipes like init and up. As the name implies, it applies the migrations to the database.

Make recipe:

migrate:
        # Run here the command to execute the migrations
Enter fullscreen mode Exit fullscreen mode

Invocation:

make check
Enter fullscreen mode Exit fullscreen mode

Check code (make check)

Any serious project run some kind of code formatting tool to comply with your company coding guidelines and best practices. This command performs these checks on the code.

Make recipe:

check:
    # Example leveraging pre-commit
    pre-commit run --all-files
Enter fullscreen mode Exit fullscreen mode

Invocation:

make check
Enter fullscreen mode Exit fullscreen mode

Bump version (make bump-version LEVEL=patch|minor|major)

This recipe updates the version of your project. Each time I commit code the pre-commit hook calls this command with the patch argument, and each time I deploy to production I manually call it with the minor|major argument depending on the type of code change I'm conducting.

Make recipe:

bump-patch:
    # Example with bump2version
    poetry run bump2version $(LEVEL)
    git add _version.py
    git add .bumpversion.cfg
Enter fullscreen mode Exit fullscreen mode

Invocation:

make bump-patch LEVEL=patch
Enter fullscreen mode Exit fullscreen mode

Run unit/acceptance tests (make run-tests and make run-acceptancetests)

This recipe runs the tests of my application with some default parameters I like to establish. Sometimes I create other similar recipes to run these very tests but with some quirks, like generating coverage and result reports when running.

Make recipe:

run-tests:
        # Call here the command to execute your tests: py.test, NUnit... just make sure that the environment variables needed by your tests are loaded. This is why I like to run my tests with docker compose!
Enter fullscreen mode Exit fullscreen mode

Invocation:

make run-tests
Enter fullscreen mode Exit fullscreen mode

Open the profiling tool (make open-profiling)

You might have been profiling some code to find bottlenecks in your application and generated a report you'd like to visualize with tools like snakeviz or pyprof2calltree. This command will open the latest report generated, ready for visualization.

Make recipe:

open-profiling:
        # Example of command, adjust to your preferred visual tool
    pyprof2calltree -i profiling/results/$(FILE).prof -k
Enter fullscreen mode Exit fullscreen mode

Invocation:

make open-profiling FILE=results
Enter fullscreen mode Exit fullscreen mode

Conclusions

I exposed several recipes that I found useful when developing. Even though I gave some clues of the tooling you could potentially use, it's impossible to provide a detailed implementation since it wildly depends on the programming language you use, the environment, etc... However, what's really important about each recipe is the main objective it tries to achieve, and that's a fact (building the project, running tests...).

I'd like to hear from other fellow developers what other recipes you have come up with that could make our lives a little bit easier. If you have any to share please, feel free to contribute in the comments section!

Discussion (0)