Python's Import System

In this blog, I want to shed some light on a somewhat overlooked and taken-for-granted feature of Python, and many programming languages for that matter, the Import System, specifically how Python goes about importing various modules and the subtle nuances involved in the process.

Before delving into the topic at hand, I want to briefly outline the difference between a Python module and a Python script, this will come in handy when we discuss relative imports.

Python Module & Python Script

In essence, both a python module and a python script are executable python code but serve different purposes, a script is a python file that is executed on its own like python xyz.py whereas a module is a python file that is imported by other python files and typically not executed on its own. Another subtle implementation difference shows up in the value of their global attribute __name__. The __name__ value for a python script is __main__ while that of a module is its path. To try it out, write an innocuous python file X.py and add the solitary line print("the __name__ attribute for X is", __name__). Now run X.py directly with python X.py and observe the output. Now, write another python file Y.py in the same directory and just add import X inside it. Run Y.py and Voila! see the difference in the value in the __name__ attribute for X.

When you first started learning to code in Python, you might have come across a statement at the end of the python program that read something like

if __name__ == "__main__":
# do something

Now you know the reason why! Python's __main__

Python Import System

For the sake of brevity, I'll confine this blog to the "basic" and "pertinent" (obviously basic and pertinent are used in relative terms, so don't take them absolutely) steps on what happens when Python executes an import XYZ statement without going into the more esoteric details, for that, you can refer Python's Import System.

From Python's official documentation, importing a module involves two stages:

• Searching the named module.
• Binding the result of the search to some name in the local namespace.

Now, the question is where does Python search the module in?
So, Python searches the required module in the following places in order:

1. sys.modules (the module cache)
2. the directories listed in sys.path (in order)

Open up a new python file and type in:

import sys
print(sys.path)

When you run this, it lists the directories Python searches to see if the requested module exists.

Another small experiment to test the precedence of imports, try this:

Create a file random.py and type in whatever you feel like it. Fire up the python shell in the same directory and import random. Which random module do you think is imported? The answer actually depends on whether you have already import random some earlier time, if yes then the in-built random module would have been cached in sys.modules and will be imported. If not then your recently created random module will be imported because the current working directory(in the interpreter it will be termed as '', an empty string) comes first in sys.path.

So, if you're stuck with an infuriating ModuleNotFoundError, try printing sys.path and append the intended module's directory in sys.path, sys.path.append('path_to_the_module_directory'). Be wary of name clashes though!

Relative Imports

This is one of the conundrums that has vexed a lot of people and mostly because the official Python documentation isn't very clear on this topic (I think). To be fair, Python's documentation does refer to the section as
Package Relative Imports and not Relative Imports and this is where we return to the distinction between modules and scripts, because again a Package is not supposed to be run on its own and anytime you call a Package you are essentially importing the __init__.py file defined in the top directory of the package.

So, here's the gist you cannot do relative imports on a file whose __name__ is __main__ because python uses this very attribute to construct paths for importing relative modules. So, if the __name__ of some file is /xyz/subpackage1/path2 then this is the base path used for any relative imports, import .moduleX basically implies to import a python module located in /xyz/subpackage1/path2 and import ..moduleY means to import moduleY from /xyz/subpackage1 (if it exists that is!)

If you're familiar with Django, its standard to see code like

from . import views

in urls.py but again we never really run urls.py on its own, we call manage.py and it implicitly calls urls.py in the background to do perform some tasks.

So there you have it, a small and hopefully definitive guide to imports in Python.

Thanks :)