DEV Community

loading...
Cover image for Python Project - Command Line

Python Project - Command Line

unfor19 profile image Meir Gabay Updated on ・6 min read

Goal

Covering the possible ways to execute a Python module (script) from the command-line (terminal).

Requirements

  1. You are familiar with Packages and Modules
  2. It would be easier to understand the examples if you run the examples by yourself - use this project as a template

Getting Started

We'll go over the possible way to execute scripts and modules from the command-line

Executing Scripts And Modules

In the following examples, all the modules are executed from the project's root directory (top-level package). This is important since the root dir has access to all the packages (sub-directories) in the project.

Executing the main.py script and appy as a module with src/appy/__main__.py

# Executing as a script - works as long as there are no relative imports of parent packages
(python-project) $ python main.py
My Path: python-project/main.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka

Hello Willy Wonka, here's the cat fact of the day:
Jaguars are the only big cats that don't roar.

# Executing as a module - works because of the __main__.py file
(python-project) $ python -m src.appy
My Path: python-project/src/appy/__main__.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka

Hello Willy Wonka, here's the cat fact of the day:
A cat has more bones than a human being; humans have 206 and the cat has 230 bones.
Enter fullscreen mode Exit fullscreen mode

Executing the module src/appy/core/app.py from the root directory

# Contains relative imports - `..utils.message`
# Excuting as a script - causes issues with relative imports of parent packages
(python-project) $ python src/appy/core/app.py
Traceback (most recent call last):
  File "src/appy/core/app.py", line 1, in <module>
    from ..utils import message, img_ascii
ImportError: attempted relative import with no known parent package

# Excuting as a module - works perfect with relative imports
(python-project) $ python -m src.appy.core.app
My Path: python-project/src/appy/core/app.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka

Hello Willy Wonka, here's the cat fact of the day:
Siamese kittens are born white because of the heat inside the mother's uterus before birth. This heat keeps the kittens' hair from darkening on the points.
Enter fullscreen mode Exit fullscreen mode

Executing the module src/appy/utils/message.py from the root dir

# Doesn't contain relative imports
# Executing as a script - no errors since there are no relative imports
(python-project) $ python src/appy/utils/message.py
My Path: python-project/src/appy/utils/message.py

# Executing as a module - works as expected
(python-project) $ python -m src.appy.utils.message
python-project/src/appy/utils/message.py
Enter fullscreen mode Exit fullscreen mode

If the PWD is a subdirectory of the project, such as python-project/src/appy, an attempt to execute a module which contains relative imports, will raise the exception below. Remember, your PWD should always be the project's root directory, in this case, it's python-project.

# PWD is `src/appy`
# Execute as a module - issues when running from a subdirectory (not from root dir)
(python-project/src/appy) $ python -m core.app

Traceback (most recent call last):
  File "/Users/meirgabay/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/meirgabay/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/meirgabay/python-project/src/appy/core/app.py", line 1, in <module>
    from ..utils import message, img_ascii
ValueError: attempted relative import beyond top-level package
Enter fullscreen mode Exit fullscreen mode

It doesn't happen when invoking message, since message doesn't use relative imports

# Execute as a script - works because there are no relative imports to parent packages, similar to main.py
(python-project/src/appy) $ python utils/message.py
My Path: python-project/src/appy/utils/message.py

# Execute as a module - works as expected, no imports of parent packages
(python-project/src/appy) $ python -m utils.message
My Path: python-project/src/appy/utils/message.py
Enter fullscreen mode Exit fullscreen mode

Invoke a function from the command-line

Trying to invoke a function from the terminal, such as appy.core.app.main(), will raise the ModuleNotFound exception. A package must be imported before invoking one of its functions.

# Execute as a module - must import the function before invoking it
(python-project) $ python -m src.appy.core.app.main
/Users/meirgabay/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/bin/python: Error while finding module specification for 'src.appy.core.app.main' (ModuleNotFoundError: __path__ attribute not found on 'src.appy.core.app' while trying to find 'src.appy.core.app.main')
Enter fullscreen mode Exit fullscreen mode

Since you can't invoke main() directly from the terminal, calling it from the if __main__ block enables executing it from the terminal. It's possible to pass arguments, but it's a bit ugly, read the docs to learn how. The following example attempts to execute the module appy.core.app, which in turn call its if __main__ block

# Execute as a module - works due to the `if __main__` block
(python-project) $ python -m src.appy.core.app
My Path: python-project/src/appy/core/app.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka

Hello Willy Wonka, here's the cat fact of the day:
Cats sleep 16 to 18 hours per day. When cats are asleep, they are still alert to incoming stimuli. If you poke the tail of a sleeping cat, it will respond accordingly.
Enter fullscreen mode Exit fullscreen mode

Invoking a function from the terminal is also possible by using the -c flag. Surprise, it's possible to pass arguments in a more intuitive way, for example src.app.main(my_arg1, my_arg2)

# Execute inline script - providing the "-c" flag with the command as a string, the semi-colon ";" means break-row
  (python-project) $ python -c "import src.appy.core.app as app; app.main()"
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka

Hello Willy Wonka, here's the cat fact of the day:
Cats can be right-pawed or left-pawed.
Enter fullscreen mode Exit fullscreen mode

What are the available command-line flags in Python?

  • In this tutorial, we used both -c and -m flags
  • Read the docs to learn about more flags - Using cmdline

Why is it possible to execute python -m src.appy?

The src/appy/__main__.py file acts like the if __main__ code snippet, but on packages. This enables the appy package to be executed with python -m or with runpy

# Execute as a module
(python-project) $ python -m src.appy
My Path: python-project/src/appy/__main__.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka

Hello Willy Wonka, here's the cat fact of the day:
One reason that kittens sleep so much is because a growth hormone is released only during sleep.
Enter fullscreen mode Exit fullscreen mode

What's runpy and why do you use it in main.py?

The runpy package provides the ability to run modules from a module (Python script). This package is shipped with Python so there's no need to install it.

main.py

import runpy


def main():
    appy_package = runpy.run_module(
        mod_name="src.appy", init_globals=globals())
    appy_package['message'].script_path(__file__)
    appy_package['main']()


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

What's globals()?

The official definition from the docs

Return a dictionary representing the current global symbol table. This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).

(python-project) $ python
Python 3.8.2 (default, Jun 30 2020, 19:04:41)
[Clang 11.0.3 (clang-1103.0.32.59)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> globals()
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>}
Enter fullscreen mode Exit fullscreen mode

This blog post is part of the Python Project series, and is based on this GitHub repository - unfor19/python-project.
The GitHub repo includes an example of a Python project, and Wiki Pages that describe the necessary steps for developing, creating and distributing a Python package.


Originally published at github.com/unfor19/python-project on 3 November, 2020

Discussion (0)

Forem Open with the Forem app