This post is brought to you in the spirit of converting tweetstorms to blogposts.
to the tweetstorm
Surprise! 🎁
In Python, if property access raises AttributeError, and the class implemented getattr, it will get called with the property name.
This results in some very cryptic errors.
If you run the following code (repl):
class Thing:
name = "Thing"
class NameProvider:
def __init__(self, name):
self.name = name
def get_name(self):
return self.nam
class ThingWrapper:
def __init__(self, thing, name_provider):
self.thing = thing
self.name_provider = name_provider
@property
def name(self):
return self.name_provider.get_name()
def __getattr__(self, name):
return getattr(self.thing, name)
def main():
thing = Thing()
name_provider = NameProvider(name="Not Thing")
thing_wrapper = ThingWrapper(thing, name_provider)
print(thing.name)
print(thing_wrapper.name)
if __name__ == '__main__':
main()
You'll get a surprising result:
Thing
Thing
You might have expected "Not Thing"
as the second line, or maybe an exception to be raised from NameProvider.get_name()
due to the typo there (self.nam
instead of self.name
). But instead, we got the name attribute from our Thing
instance.
Analysis 🔎
If you've every used __getattr__()
you know that it is called when the named attribute was not found using other lookup mechanisms. That said, it might not be clear to you that this includes properties raising AttributeError
exceptions. It definitely wasn't clear to me.
That is, it was unclear to me despite being clearly stated in the documentation for getattr()
object.__getattr__(self, name)
Called when the default attribute access fails with anAttributeError
(either__getattribute__()
raises anAttributeError
because name is not an instance attribute or an attribute in the class tree for self; or__get__()
of a name property raisesAttributeError
). This method should either return the (computed) attribute value or raise anAttributeError
exception.
Beside being surprising, there are 2 main issues here:
- Any code down the stack from the property can effectively change attribute lookup for the class by throwing an
AttributeError
. In the above example - a typo inNameProvider
caused an attribute to be taken fromThing
instead, against the programmer's obvious intention. - The exception is silenced. There is no way for the programmer to catch the exception outside the property getter. This makes the errors very hard to track down. This also means that whenever you add
__getattr__()
to a class, you're silencing allAttributeError
exceptions that were previously thrown from properties.
Like anything in Python, you can hack around the issue. In this case - with fancy decorators!
Solution? 🐍
Consider the following code (repl):
class ExceptionCatcher:
def __init__(self, f):
self.f = f
self.exception = None
def __call__(self, *args, **kwargs):
try:
return self.f(*args, **kwargs)
except Exception as e:
self.exception = e
raise
else:
self.exception = None
def store_exception(f):
return ExceptionCatcher(f)
def load_exception(f):
def _raise_property_exception(instance, name):
try:
class_attr = getattr(instance.__class__, name)
if not isinstance(class_attr, property):
return
exception = class_attr.fget.exception
except AttributeError:
return
if exception:
raise exception
def _wrapper(*args, **kwargs):
_raise_property_exception(*args, **kwargs)
return f(*args, **kwargs)
return _wrapper
class ThingWrapper:
def __init__(self, thing, name_provider):
self.thing = thing
self.name_provider = name_provider
@property
@store_exception
def name(self):
return self.name_provider.get_name()
@load_exception
def __getattr__(self, name):
return getattr(self.thing, name)
If you run this version, you'll get the following exception:
AttributeError: 'NameProvider' object has no attribute 'nam'
This matches our expectations far better.
This result is achieved in two steps. First, we store all the exceptions thrown from name()
so that we can throw them again if needed. Then, before calling __getattr__()
, we check if we got there due to a property raising an exception. If we did - we just re-raise that exception.
The rest is implementation details, and I probably missed something there (you might notice that I corrected a bug when converting the tweets to this post - in the previous version, I forget to reset the exception storage after successful property retrieval).
While this solution works, and may be useful for detecting similar bugs, I would probably avoid using it in production code. Instead, I'd be happy to have some standard Python construct to provide this functionality.
Top comments (0)