Serialization fails with no attribute __qualname__

Description

Serialization fails when a task uses an instance of a callable class as a cache_validator.

AttributeError                            Traceback (most recent call last)
~/.virtualenvs/iguazu-venv/lib/python3.7/site-packages/prefect/utilities/serialization.py in _serialize(self, value, attr, obj, **kwargs)
    379         try:
--> 380             qual_name = to_qualified_name(value)
    381         except:

~/.virtualenvs/iguazu-venv/lib/python3.7/site-packages/prefect/utilities/serialization.py in to_qualified_name(obj)
     36     """
---> 37     return obj.__module__ + "." + obj.__qualname__
     38

AttributeError: 'Validator' object has no attribute '__qualname__'

During handling of the above exception, another exception occurred:

... 

~/.virtualenvs/iguazu-venv/lib/python3.7/site-packages/prefect/utilities/serialization.py in _serialize(self, value, attr, obj, **kwargs)
    383                 raise ValidationError("Invalid function reference: {}".format(value))
    384             else:
--> 385                 return qual_name
    386
    387         # sort matches such that the longest / most specific match comes first

UnboundLocalError: local variable 'qual_name' referenced before assignment

More details:

I found a small bug on the serialization code. In prefect.utilities.serialization.StatefulFunctionReference :

    def _serialize(self, value, attr, obj, **kwargs):  # type: ignore
        if value is None and self.allow_none:
            return None

        try:
            qual_name = to_qualified_name(value)
        except:
            if self.reject_invalid:
                raise ValidationError("Invalid function reference: {}".format(value))
            else:
                return qual_name
        ... more code ...

There is a return with an undefined variable (because when this fails, there is no qual_name variable.

I am not sure if this is a problem of that particular function, or is from to_qualified_name, which fails with an input instance instead of a class. The culprit from my side is that I have a task with cache_validator=SomeClass() … where SomeClass is a class with a __call__ member function, instead of using a cache_validator that is a function. I am attaching an example on the reproduction section.

Expected Behavior

Since cache_validator in a task is type-annotated to a Callable, I expected to be able to use an instance of a callable class without issues when serializing.

Reproduction

import datetime

from prefect import Flow, Task
from prefect.environments.storage import Docker


class Validator:
    def __init__(self):
        print('I am a validator, ready to say false al the time')
    def __call__(self, state, inputs, parameters):
        return False

my_task = Task(cache_validator=Validator(), cache_for=datetime.timedelta(days=1))
with Flow('example') as flow:
    my_task()

flow.storage = Docker()
flow.register()

Environment

$ prefect diagnostics
{
  "config_overrides": {},
  "env_vars": [],
  "system_information": {
    "platform": "Darwin-19.3.0-x86_64-i386-64bit",
    "prefect_version": "0.10.4",
    "python_version": "3.7.5"
  }
}

[jcrist]

Independently from our workaround / refactor, I think the undefined variable is still a bug, don’t you think?

Definitely. I’ve added a PR at #2491 to provide a better error message in this case. Your example now raises:

ValidationError: Invalid function reference, function required, got <__main__.Validator object at 0x10a892050>

[jcrist]

With #2491 we raise a better error message. We plan in the future to relax this restriction, but it seemed prudent to give a more meaningful error in the meantime. See #2499 for future work.