[Feature Request] PEP 612 support

Hey bear~d~ lovers! ^{_{This was definitively not an obscure reference}}

The start of a journey

The other day I designed a decorator that adds one parameter to a function. I know the following example is silly but bear with me:

def with_secret(fct):
    def wrapped(*args, **kwargs):
        return fct("p4ssw0rd", *args, **kwargs)
    return wrapped

def guess_size(secret, size):
    return len(secret) == size

assert guess_size(6) == False
assert guess_size(8) == True

With type hints

Adding the extra parameter as first positional parameter was no coincidence: I am limited by PEP 612. Which gives the following code with type hints everywhere:

from typing import Callable, Concatenate, ParamSpec, TypeVar

P = ParamSpec("P")
T = TypeVar("T")

def with_secret(fct: Callable[Concatenate[str, P], T]) -> Callable[P, T]:
    def wrapped(*args: P.args, **kwargs: P.kwargs) -> T:
        return fct("p4ssw0rd", *args, **kwargs)
    return wrapped

@with_secret
def guess_size(secret: str, size: int) -> bool:
    return len(secret) == size

Mypy is happy, therefore I am. Rejoice!

Where I try to fix things

However, inspect.signature as well as typing.typing.get_type_hints look sad, therefore I am.

>>> signature(guess_size)
<Signature (*args: P.args, **kwargs: P.kwargs) -> ~T>
>>> get_type_hints(guess_size)
{'args': P.args, 'kwargs': P.kwargs, 'return': ~T}

Adding functools.wraps is a good step forward, but the extra parameter appears:

>>> signature(guess_size)
<Signature (secret: str, size: int) -> bool>
>>> get_type_hints(guess_size)
{'secret': <class 'str'>, 'size': <class 'int'>, 'return': <class 'bool'>}

I cry, then persevere:

from functools import wraps
from inspect import signature

def with_secret(fct: Callable[Concatenate[str, P], T]) -> Callable[P, T]:
    @wraps(fct)
    def wrapped(*args: P.args, **kwargs: P.kwargs) -> T:
        return fct("p4ssw0rd", *args, **kwargs)

    sig = signature(fct)
    parameters = tuple(sig.parameters.values())

    # fix signature
    # see https://github.com/python/mypy/issues/12472 about the ignore below
    wrapped.__signature__ = sig.replace(  # type: ignore
        parameters=parameters[1:],
    )

    # fix annotations
    try:
        del wrapped.__annotations__[parameters[0].name]
    except KeyError:
        pass  # maybe the first parameter of fct has no type hint

    return wrapped

Now things seems better:

>>> signature(guess_size)
<Signature (size: int) -> bool>
>>> get_type_hints(guess_size)
{'size': <class 'int'>, 'return': <class 'bool'>}

Let the bear in

And now for the grand finale I introduce our bear friend:

from beartype import beartype

@beartype
@with_secret
def guess_size(secret: str, size: int) -> bool:
    return len(secret) == size

Alas! The bear is hibernating and does not react to the following calls:

>>> guess_size(None)
False
>>> guess_size("wrong type")
False

Note that changing the decorators order (beartype/with_secret) seems to work but prevents use cases such as the beartype decorator being added automatically.

<div align="center">

</div>

At this point my guess is that the bear needs understanding and sympathy, and my decorator needs some more trickery.

Could you give a little help in awaking the bear so that invalid calls are roared at? 🐻💬

[leycec]

…country known for its 2018 indie platformer…

A fellow gentleman of culture, I see. I instinctively knew that you (of course) referred to the modern monument to Canadian suffering that is Celeste – despite having never played Celeste. Why? Because… this. is. Canada.

Horrifying West Coast mountain ranges literally embodying our constant struggle with wintertime depression are in our collective DNA here. Although I myself grapple with PEP Standards Angst, ^{pretty sure it’s a thing} my Dad wrestled the shadowy demon of depression his entire adult life.

In honour of him, Celeste is always a thing I wanted to 100% – B-sides, C-sides, and all. This 1:35-minute run of the Chapter 7: The Summit C-side gets me every time. There is anguish implicit in that run.

When I was a mere scruffy ruffian, only Mega Man 2 Wily’s Fortress: Stage 3 was well-known by pale-faced, thin-wristed millennials the world over to harbour such devilish wall spike level design yet unmatched by mankind. Like, seriously – that cluster of three fall-into-narrow-wall-spike-chimney screens at the 1:00-minute mark still engenders a nostalgic shudder of bittersweet hatred “joy.”

Then Celeste happened and invalidated all my childhood. 💢

…have you tried getting in touch with the development of the Python language?

You presume too much! I jest, of course. Alas, Guido and whatever resident Python BFDL is currently wearing the Guido Mask don’t necessarily align with runtime type-checking. Python’s established dev culture isn’t exactly opposed to runtime per say – but ignoring runtime kinda does constitute a kind of opposition, doesn’t it? It’s like a “lie by omission.” Technically, that’s not a lie. Pragmatically, that’s pretty much a lie.

It’s true. I could vociferously argue while pounding my fists in the general direction of the Steering Council. But over the last tumultuous 40-oddball years, I’ve learned that humans don’t particularly respond to logical argumentation that contradicts cherished assumptions; humans only respond to action. Therefore, I act. I contradict cherished assumptions by creating something that, by all accounts, shouldn’t exist.

That’s actually how I derive most of my fun here. I’m anti-authoritarian by nature. Honestly, I blame childhood poverty in Los Angeles. That’s pretty much the breeding ground for anti-authoritarian dispositions right there.

So, I love doing things I shouldn’t. Runtime type-checking in Python is a thing no one should do, because:

• It feels NP-hard – and probably is.
• It feeds on continual dark magic. Implementing standards never intended to be implemented requires:
  • Exploiting deep flaws in the CPython runtime.
  • Violating privacy encapsulation in the standard library.
  • Circumventing perpetual roadblocks with eval-based Aikido throws.
  • Giggling incoherently to myself as tests pretend to finally pass.
• It means subverting community expectations, standards bodies, and multinational corporations – all of which have gone all-in on static type-checking to the exclusion and detriment of runtime type-checking.
• There’s utterly no fame, fortune, or tangible incentive in it (because prior point).

Like, seriously. The Big Three (so: Google, FaceBook Meta, and Microsoft) each created their own ad-hoc unofficial third-party static type-checker for Python – none of which work particularly well and all of which argue with one other and Python’s official first-party static type-checker mypy about everything. Really? Nobody bothered to ever consider runtime, even though it’s ultimately only runtime and not “static analysis time” that matters. Really? Yet, it happened. It’s sorta unbelievable when you put it in that deadpan historical context.

Basically, it’s a one-bear war against QA injustice that we prosecute here. That framing may sound obnoxiously unhealthy (and probably is), but it’s the funn(i)est thing I’ve ever done. Indeed! It’s the only thing I’ve ever done that’s consistently captured my undivided attention without any expectation of extrinsic reward.

@beartype is 100% intrinsic reward. We’re here because we make code better, despite the world. Fun rapidly ensues. Roar, bear. Roar! 🐻

…looking back to the related Pydantic case makes me think that maybe things would be for the better if dynamic-typing library authors were more involved in PEPs & stuff.

You’re absolutely right. Pydantic maintainers and collaborators became heavily involved, averting the obvious pending crisis that was obvious to everyone except CPython devs.

But… Pydantic’s popular. Pydantic fuels FastAPI (ironically, despite being slow-as-molasses), which fuels over 230 downstream dependencies. These include friggin’ Pycharm, which comes bundled with a FastAPI project type out-of-the-box.

Meanwhile, @beartype quietly lumbers about in the background. We’re over here snuffling beehives, stuffing our face full of wild blueberries, and thrashing about with wild-caught salmon in the vast Sockeye runs of British Columbia.

But, really? I just don’t like arguing with people. That’s the core low-level issue here. Ultimately, it’s me. Can’t deflect blame on that one. You know that Little Simz mic-drop Sometimes I Might Be Introvert from a year or so ago? Let’s pretend you do. Well…

I like violating expectations by implementing things at runtime. The human side of that equation is equally critical – but it repulses me, because always I am introvert.

I am in no hurry of having PEP 612 supported by @beartype…

😌

…as I’m not expecting to deploy impacted code anywhere for at least a few months.

😥

Okay. I tell you what we are going to do.

For you, we’ll do something. PEP 612 is a bit beyond even the immediate horizon of the next several months. But fear not! I see that you are munging the PEP 362-compliant func.__signature__ dunder instance variable up above:

    wrapped.__signature__ = sig.replace(  # type: ignore
        parameters=parameters[1:],
    )

That’s wonderful. Surprisingly, that’s standardized. Equally surprisingly, that requires no risky call stack inspection on @beartype’s part. Instead, @beartype should be mildly refactored to obey the algorithm outlined by PEP 362:

1. If the callable being decorated by @beartype defines the func.__signature__ attribute, @beartype should parse arguments from that signature. This should be mostly trivial. Rejoice! The inspect.Signature API is object-oriented, sane, and readable.
2. Else, @beartype should parse arguments from the code object underlying the decorated callable. We already do this. Rejoice again!

Rejoice is what I’m saying. Oh, and gently prod me if I neglect this. You need this, so I need and want to do this. Also, it sounds fun. Right? Let’s do even more things we’re not supposed to, everyone! 🥳

[leycec]

Belated reply arrives… belatedly.

Let’s admit this has been a busy summer and we’ll agree to leave it at that. Thankfully, tomorrow is lakeside kayaks and trailside bicycles under a baking hot sun all day. Thank you, sun. We give thanks in Canada for your infrequent existence. 🌞

In other news…

So is this the automatic decoration of entire modules the only thing that is holding @beartype back from full support of PEP 612?

Depends on what we mean by “full support of PEP 612,” right?

Ideally, that should mean actual support for deep runtime type-checking typing.ParamSpec(...) and typing.Concatenate[...] type hints. Pragmatically, the convenient fact that @beartype type-checks at runtime does kinda mean we can wildly wave our hands around and pretend we’re doing something we’re actually not.

Actual PEP 612 support is definitely still on the plate. But PEP 362 support via the func.__signature__ dunder variable is deliciously easier and achieves similar (but arguably even more useful) goals… So, we’ll probably run over that goalpost first.

Until then, please reorder your decorator chains is what I’m saying. @beartype demands precedence, because it’s jerkish. At least @beartype isn’t snuffling around in your backyard composter – unlike some bears and raccoons I could mention. :face_exhaling:

The Shape of What Is to Come

Incidentally, I’ve fully implemented (but have yet to actually test) import hooks.

Once this lands in beartype 0.11.0 or 0.12.0, you’ll be able to trivially @beartype all callables and classes across your entire app stack or Python process with this two-liner at the top of your {package_name}.__init__ submodule:

# Congrats. You just @beartyped literally everything. UwU
from beartype.claw import beartype_all
beartype_all()

Thanks as always, everyone, for being special. You’re all awesome! You drive us to better @beartype and thereby better ourselves. 🧘‍♂️ ^{⇐ you after @beartype}