Python 3.9: What's new?

Python 3.10 is coming but have you tried 3.9? In this article I'll show you cool new features introduced to the current version of the Python programming language and give you reasons why you ought to be cautious when porting to newer version

With every Python update comes new, deprecated and obsolete features. Python 3.9 was officially released on October 5th, 2020. With 3.10 on the way, version 3.9 is Python's biggest update yet. This article will be showing the cool features, and modules introduces to the language and those that are no longer supported.

New features.

Here are the cool new features introduced in the current version. These are feature update to dictionary, string, type hinting and a new language parser.

New dictionary operators.

Python's built-in dict class comes with a new way of merging updating dictionary. The updates compliment the existing methods of merging dictionaries using thedict.update and {**d1, **d2}.

>>> set_x = {"key1": "value1 from x", "key2": "value2 from x"}
>>> set_y = {"key2": "value2 from y", "key3": "value3 from y"}

>>> set_x | set_y
{'key1': 'value1 from x', 'key2': 'value2 from y', 'key3': 'value3 from y'}

>>> {**set_x, **set_y} # The same as the above
{'key1': 'value1 from x', 'key2': 'value2 from y', 'key3': 'value3 from y'}

From the above, you can see that set_x | set_y is the same thing as {**set_x, **set_y}. on closer inspection, you may notice that the merge is made based on the dictionary key. This is because the operator calculate the element in the dictionaries as a union of sets.

The new merge operator can be be used to merge different dictionary-like object and preserve the object. The {**d1, **d2} method will rather return a dict class.

>>> from collections import defaultdict

>>> fruits = defaultdict(
    lambda: None, 
    {"cabbage":"60% Vitmain C", "carrot":"5% Vitamin C", "broccoli":"70% Vitamin C"}
)

>>> Veggies = defaultdict(
    lambda: None, 
    {"apple":"7% Vitamin C", "mango":"60% Vitamin D", "orange":"88% Vitamin C"}
)

>>> fruits | veggies
defaultdict(<function <lambda> at 0x000002379439DF70>, 
    {'cabbage': '60% Vitmain C', 'carrot': '5% Vitamin C', 'broccoli': '70% Vitamin C', 'apple': '7% Vitamin C', 'mango': '60% Vitamin D', 'orange': '88% Vitamin C'})

>>> {**fruits, **veggies}
{'cabbage': '60% Vitmain C', 'carrot': '5% Vitamin C', 'broccoli': '70% Vitamin C', 'apple': '7% Vitamin C', 'mango': '60% Vitamin D', 'orange': '88% Vitamin C'}

Update using the |= operator.

>>> ds_libraries = {
...    "numpy": "Mathematical functions",
...    "Pandas": "Data structure manipulations",
...    "Scipy": "Scientifc calculations"
}

>>> ds_libraries |= {"Matplot": "Data visualization"}
>>> ds_libraries
{"numpy": "Mathematical functions", "Pandas": "Data structure manipulations", "Scipy": "Scientifc   calculations", "Matplot": "Data visualization"}

The new update operator can be used to update data types with a dictionary-like data structure, unlike the merge (|) operator where the two dictionary must be of proper types.

>>> #...

>>> # List of tuples
>>> ds_libraries |= [("","")]
>>> ds_libraries

For more information about the new dictionary operators, see PEP 584.

New string methods

New string methods have been added to remove the beginning (prefix) and end (suffix) of a string. Introduced here are the removeprefix() and removesuffix() methods

Say you wan to remove the prefix "Python" in the string "Python 3.9 is cool." You'd use the removeprefix method. The syntax is str.removeprix(prefix).

>>> string = "Python 3.9 is cool"
>>> string.removeprefix("Python")
' 3.9 is cool'

same thing goes for the suffix. The syntax is str.removesuffix(suffix)

>>> string = "Python 3.9 is cool">>> string.removesuffix("cool")'Python 3.9 is '

If the given prefix or suffix of the string passed to either methods is empty, then the string will be returned.

>>> string = "Python 3.9 is cool"
>>> string.removesuffix("")
"Python 3.9 is cool"
>>> string.removesuffix("")
"Python 3.9 is cool"

Also, either methods only remove one copy of the prefix or suffix. To remove all, use a while loop

>>> string = "Laracoco"
>>> string.removesuffix("co")

>>> while string.endswith("co"):
...     string.removesuffix("co")
...
>>>
>>> string
"Lara"

For more information about the new string methods, see PEP 616.

Built-in type hinting.

Before Python 3.9, creating a collection or generic type for code annotation needed importing the typing module. A collection/generic type is a container that can be filled with other types, such as a list of dictionaries. In previous versions of python, this couldn't be achieved. It wasn't possible to use list[dict] or list[float] to type hint, you'd rather use a corresponding capitalized type from the typing module.

from typing import List

def greeting(names: List[str]) -> None:
    for name in names:
        print("Hi", name)

In Python 3.9, using the imported List and Dict corresponding generic type is eliminated. Python now has them built into the language for easy type hinting.

def greeting(names: list[str]) -> None:
    for name in names:
        print("Hi", name)

New Parser

A core part of the Python interpreter is the parser.

To parse means to resolve data into a structure that describe it meaning.

Simply put (In the way even I would understand), A parser is a component of a programming language's interpreter or compiler that translate lines of code into machine-readable language.

For Python, the parser organizes the code in tree-like manner known as a parse-tree or Abstract Syntax Tree (AST) so the interpreter can convert to native code.

Python 3.9 comes with reimplemented parser that is based on Parsing Expression Grammar (PEG) instead of Left-to-left, Leftmost derivation (LL Parser).

The performance of the new parser is no much different from the old one. However, "the PEG formalism is more flexible than LL when it comes to developing new language features." That's to say that much hack around the new parser is not needed as it was the case with the old parser that needed circumventing. PEG parser will work in the same way as the old one by producing Abstract Syntax Tree (AST) for the interpreter.

The new Python update ships with both parser, but has PEG as it default. Having both parsers is good for comparing performance at the AST level.

The old parser will be remove in the next version of Python, version 3.10. Doing so will allow for development of new language features without the limitation of the old parser.

See PEP 617 for
more information.

New Modules.

Version 3.9 introduced two modules: Zoneinfo for time zone support and graphlib for sorting graphs.

Proper time zone with zoneinfo

Previous version of Python lacked built-in support for working with time zones (technically, it did, but only supported UTC), to do so, you'd have to used the 3rd party dateutil module.

The new zoneinfo is not only built-in but makes working with easy. The module introduce support for the Internet Assigned Number Authority (IANA) time zone database to the standard library.

You can access time zone by using zoneinfo.ZoneInfo and passing in country/state [or city] key. It'll return a description of the time zone as an object.

from zoneinfo import ZoneInfo

>>> ZoneInfo("Africa/Lagos")
zoneinfo.ZoneInfo(key='Africa/Lagos')

NOTE

If you get an error as the following:

ModuleNotFoundError: No module named 'tzdata'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python39\lib\zoneinfo\_common.py", line 24, in load_tzdata
    raise ZoneInfoNotFoundError(f"No time zone found with key {key}")
zoneinfo._common.ZoneInfoNotFoundError: 'No time zone found with key Nigeria/Lagos'

That indicate there's no local IANA time zone database in the machine, hence zoneinfo is unable to locate it. To solve the error, install the tzdata, which is a module containing the IANA time zone database.

>>> python pip install -m tzdata

Using zoneinfo.ZoneInfo, you can create a timestamp with proper time zone.

>>> from datetime import datetime
>>> from zoneinfo import ZoneInfo
>>>
>>> timezone = ZoneInfo("Africa/Lagos")
>>> dt = datetime.now(tz=timezone)
>>> dt
datetime.datetime(2021, 8, 16, 10, 59, 1, 937856, tzinfo=zoneinfo.ZoneInfo(key='Africa/Lagos'))
>>> dt.tzinfo # Show time zone description
zoneinfo.ZoneInfo(key='Africa/Lagos')
>>> dt.tzname() # Show time zone name
'WAT'
>>>

See PEP 615 for me information.

Graph topological sorting using graphlib.

For sorting data represented as a graph, Python 3.9 adds a new module graphlib into the standard library and it contains the graphlin.TopologicalSorter class to offer functionality to perform topological sort of graph.

For instance, packages installed from the PyPi repository may have dependencies that also, in turn, may have other dependencies. This forms a structure of nodes and links (edges), each package is a node and each dependencies a link.

If you wanted to install a package that has dependencies, and you want to make sure all other related dependencies are installed, you'd do a topological sort to find the total order of required dependencies.

With the graphlib module, you can perform a topological sort to find the total order of dependencies as a list. For instance, you can represent a package and its dependencies as dictionary.

dependencies = {
    "package_A": ["depedency_A", "depedency_B"],
    "dependency_B": ["dependency_F"],
}

The dictionary represents all the dependencies of package_A, and from inspection, Package_A depends on dependency_A and depedency_B. depedency_B depends on dependency_F. The actual dependencies of package_A are depedency_A and depedency_B.

To calculate the total order of the graph to know which dependencies should be installed first, use the TopologicalSort class from graphlib.

>>> from graphlib import TopologicalSorter
>>> #...
>>> ts = TopologicalSorter(dependecies)
>>> list(ts.static_order())
['depedency_A', 'depedency_B', 'dependency_F', 'package_A', 'dependency_B']

The order above, given as a list, suggest that depedency_A should be installed first, followed dependency_B, then dependency_F, followed by the package and lastly, dependency_B.

NOTE:

I don't claim to have an extensive(it is rather shallow) knowledge about Topological ordering and dependency resolution. See the following for more: Topological sorting - Wikipedia, Total order - Wikipedia, https://en.wikipedia.org/wiki/Dependency_(disambiguation).

For more information graphlib module and its API, see the documentation.

Other updates.

In python 3.9, many modules have been improved upon. The http module added two new status code, imaplib modules added optional timeout parameters for their constructors, math module has expanded the math.gcd() function handle multiple argument and so on.

The list is quite extensive, see here for more.

Deprecated code.

Many code modules and function to the standard library have been deprecated, notably the support for Python 2.7 code.

Some Python 2.7 deprecated that's been support for backward compatibility have been removed, and many more will be removed in Python 3.10.

For more information on deprecated code, see deprecated warnings and deprecated code.

Upgrading to Python 3.9.

You might be tempted to upgrade to version 3.9, if that desire test out the new feature then yes, go head. You can install it using a virtual environment (venv) so it sits side-by-side with your current Python version. If it's what I'm thinking then you should think twice.

You're current project in production or development running on version 3.7 or 3.8 may not need any changing, if you insist, think these: Should I upgrade my project environment and make it dependent on Python 3.9?

Moving to python 3.9 (or any newer version) will benefit you and/or your team if the project environment under control and will not affect other users or the community. You can go ahead and use all the cool new features available.

You shouldn't port to a newer version if it'll affect the community, a package library written in version 3.7 will lost some function that have been deprecated in 3.9. Changes to community project should be done cautiously and gradually.

See porting to Python 3.9 for more.

Python 3.10?

Yes, that's right! as at the time of this writing, python 3.10 should be release next month October 25, 2021.

After the release of the current version, going forward, all version will be release every 12 months, steering away from the usual eighteen months release cycle.

With faster release cycle, the Python language will release features faster with fewer changes, however, it won't get new features faster or become incompatible faster. All new release will receive support for five years. The current version, 3.9, after its initial release will receive support till 2025.

Python 3.10 will be special because it's the first Python version with a double digit minor version (.10)

Conclusion

As snake shed their skin so will the Python programming language. The current release is the most exciting yet. In about a month from now, it'll shed again. But that doesn't mean version 3.9 will die off, it's not even as widespread as 3.7 was. It usually takes a while before community project fully port to newer versions.

Go ahead and try the listed features out, it might just fit right into what you're building or about to build.

Tweet at me. let's connect.

cheers!