DEV Community

Cover image for Test and Publish a Python Package to PyPi using Poetry and GitHub Actions
Ian Cleary (he/him)
Ian Cleary (he/him)

Posted on

Test and Publish a Python Package to PyPi using Poetry and GitHub Actions

My Workflow

I created a python package cookiecutter showcasing two GitHub actions workflow files I've seen and used successfully to test and publish a Python package to PyPi using GitHub actions.

The detailed explanation and python cookiecutter wrapper are the novel additions. ๐ŸŽ‰ I found it difficult to just inspect open source projects work and put the pieces together.

So I hope this helps you (no matter your previous experience).

Please let me know what isn't clear!

Getting started with this template

There are two workflow files, each with their named purpose:

  • Test
  • Publish

If you have an existing python project or are starting a new one, you can use your GitHub username and your project name as inputs into my Cookiecutter template!

First, what is cookiecutter?

Cookiecutter

GitHub logo cookiecutter / cookiecutter

A command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, VueJS projects.

Cookiecutter

pypi python Build Status codecov slack docs Code Quality

A command-line utility that creates projects from cookiecutters (project templates), e.g. creating a Python package project from a Python package project template.

Cookiecutter

We are proud to be an open source sponsor of PyCon 2016.

Features

Did someone say features?

  • Cross-platform: Windows, Mac, and Linux are officially supported.
  • You don't have to know/write Python code to use Cookiecutter.
  • Works with Python 3.6, 3.7, 3.8, 3.9 and PyPy3.
  • Project templates can be in any programming language or markup format Python, JavaScript, Ruby, CoffeeScript, RST, Markdown, CSS, HTML, you name it You can use multiple languages in the same project template.
  • Simple command line usage:
# Create project from the cookiecutter-pypackage.git repo template
# You'll be prompted to enter values.
# Then it'll create your Python package in the current working directory,
# based on those values.
$
โ€ฆ
Enter fullscreen mode Exit fullscreen mode

Make sure you have Python3 installed!

Instructions to use iancleary/pypackage

python3 -m pip install cookiecutter
# you can add the flag `--user` if you'd like

$ cookiecutter https://github.com/iancleary/pypackage
# For the sake of brevity, repos on GitHub can just use the 'gh' prefix
$ cookiecutter gh:iancleary/pypackage
Enter fullscreen mode Exit fullscreen mode

Features

  • Poetry (virtual environment and publish to PyPi, all with one tool)
  • black (linting/formatter)
  • autoflake (removing unused packages)
  • isort (dependency organization)
  • mypy (static type checking)
  • pytest (including test coverage)
  • pre-commit (hooks on commit)
  • GitHub Actions for CI/CD
  • mkdocs for documentation (with material theme)

Only Python 3.6+ is supported as required by the black, pydantic packages

Submission Category: Maintainer Must-Haves

Yaml File or Link to Code

GitHub logo iancleary / pypackage

Cookiecutter python package using Poetry, mypy, black, isort, autoflake, pytest, mkdocs, and GitHub Actions

ARCHIVED, pypackage

I no longer with to maintain this cookiecutter, please see br3ndonland/template-python's template. While not a cookiecutter at the time of this writing, it has the tooling I was going for :)

Rapid python package setup with modern venv, dependencies, testing, docs, and CI

Test Publish Dependabot Enabled

The Basic Idea

This is a template module collecting many utilities I have liked from other projects, to serve as a personal reference.

Getting started with this template

pip install --user cookiecutter
$ cookiecutter https://github.com/iancleary/pypackage
# For the sake of brevity, repos on GitHub can just use the 'gh' prefix
$ cookiecutter gh:iancleary/pypackage
Enter fullscreen mode Exit fullscreen mode

Features

  • Poetry (virtual environment and publish to PyPi, all with one tool)
  • black (linting/formatter)
  • autoflake (removing unused packages)
  • isort (dependency organization)
  • mypy (static type checking)
  • pytest (including test coverage)
  • pre-commit (hooks on commit)
  • GitHub Actions for CI/CD
  • mkdocs for documentation (with material theme)

Only Python 3.6+ is supported asโ€ฆ

Test

name: Test

on:
  push:
  pull_request:
    types: [opened, synchronize]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.6, 3.7, 3.8]
      fail-fast: false

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v1
        with:
          python-version: ${{ matrix.python-version }}
      - name: Get full python version
        id: full-python-version
        run: echo ::set-output name=version::$(python -c "import sys; print('-'.join(str(v) for v in sys.version_info))")
      - name: Install poetry
        run: |
          curl -fsS -o get-poetry.py https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
          python get-poetry.py -y
          echo "::set-env name=PATH::$HOME/.poetry/bin:$PATH"
      - name: Configure poetry
        run: poetry config virtualenvs.in-project true
      - name: Set up cache
        uses: actions/cache@v1
        id: cache
        with:
          path: .venv
          key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.version }}-${{ hashFiles('**/poetry.lock') }}
      - name: Ensure cache is healthy
        if: steps.cache.outputs.cache-hit == 'true'
        run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
      - name: Install Dependencies
        run: poetry install
      - name: Test
        run: poetry run bash scripts/test.sh
      - name: Upload coverage
        uses: codecov/codecov-action@v1
Enter fullscreen mode Exit fullscreen mode

Publish

Repo Secrets

Go to your repo settings and add a PYPI_TOKEN environment variable:

GitHub Actions PYPI_TOKEN secrets visual aid

name: Publish

on:
  release:
    types:
      - created

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v1
        with:
          python-version: "3.7"
      - name: Install poetry
        run: |
          curl -fsS -o get-poetry.py https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
          python get-poetry.py -y
          echo "::set-env name=PATH::$HOME/.poetry/bin:$PATH"
      - name: Configure poetry
        run: poetry config virtualenvs.in-project true
      - name: Set up cache
        uses: actions/cache@v1
        id: cache
        with:
          path: .venv
          key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.version }}-${{ hashFiles('**/poetry.lock') }}
      - name: Ensure cache is healthy
        if: steps.cache.outputs.cache-hit == 'true'
        run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
      - name: Install Dependencies
        run: poetry install
      - name: Publish
        env:
          PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
        run: |
          poetry config pypi-token.pypi $PYPI_TOKEN
          bash scripts/publish.sh
Enter fullscreen mode Exit fullscreen mode

That's it!

When you make a release on GitHub, the publish workflow will run and deploy to PyPi! ๐Ÿš€๐ŸŽ‰๐Ÿ˜Ž

Additional Resources / Info

I have used the cookiecutter successfully in my FastRF project

GitHub logo iancleary / fastrf

RF Design Server

fastrf

FastRF is an application to create and track key system metrics

Test Publish Dependabot Enabled Coverage Package version Python Versions

The Basic Idea

Fastrf is a web server that makes it easy to create radio frequency (RF) gain line ups ๐Ÿ“ก, link budgets ๐ŸŒŽ๐Ÿ“ก๐Ÿ›ฐ๏ธ, and simulation models ๐Ÿงช๐Ÿงฎ to caputre the performance of a RF system.

This project will stand on the shoulders of some giants:

Installing fastrf

Install the latest release:

pip install fastrf
Enter fullscreen mode Exit fullscreen mode

Documentation

Documentation is hosted at fastrf.org.

Dependencies

The pyproject.toml file is used by poetry to install dependencies into a virtual environment.

Poetry Command Line Interface Documnetation: https://python-poetry.org/docs/cli/

Only Python 3.6+ is supported as required by the black, fastapi, pydantic packages

This package was created with the iancleary/pypackage cookiecutter.


Source Link for Banner Image: https://medium.com/swlh/automate-python-testing-with-github-actions-7926b5d8a865

Top comments (0)