Numeric Tower / Union[float, int]

I think I found an edge case with the annotation float when passed an int in beartype. For convenience, PEP484 accepts int when float is annotated: https://www.python.org/dev/peps/pep-0484/#the-numeric-tower

from beartype import beartype


@beartype
def measure_cave(length: float, width: float):
    return length * width


measure_cave(12.0, 12.0)  # PASS
measure_cave(12, 12)  # type hint <class 'float'>, as "12" not instance of float.

I’m currently using Union[float, int] to work with beartype and wondered if you would consider implementing the PEP484 “numeric tower” logic or keep beartype as-is?

[posita]

⚠️ THREADJACK RISK ⚠️ - Consider ignoring this comment and proceeding as if it didn’t exist.

Ah, yes. The good ol’ numeric tower. A source of perfection and clarity and zero controversy and not at all a half-finished-but-advertised-as-complete home for all things number-related. Consider:

Personally I think PEP 3141 is a dead end. Very few people import the numbers module to use with isinstance() or issubclass() and even fewer use it to implement their own numeric types; and very few people writing numeric code care whether their code works for numbers that aren’t instances of the concrete classes they themselves use regularly.

—gvanrossum, 2017

IMO PEP 3141 was a flight of fancy and has never produced much useful effect. Pythoneers are generally to close to practice to care about the issue, most of the time the code just works through the magic of duck typing, and when it doesn’t people typically say “well that’s too fancy anyways”. 😃

—gvanrossum, 2017

So I stick to my observation that the numeric tower is still the best way to check for a specific type of number, since all builtin types are registered as member of the appropriate level in the tower.

—gvanrossum, 2020

So at least that guy is slowly coming around. 👌😅

I will encourage @leycec to weigh in here regarding the specific issue of treating int as a subclass of float within beartype, not only because this is his show, but because his mastery far exceeds my own. However, if you want native number compatibility as well as compatibility with the numeric tower generally, that’s probably an effort that you will struggle with beyond beartype.

This work-around may help you if you need something right now:

from operator import __mul__
from typing import SupportsFloat
from beartype import beartype

@beartype
def measure_cave(length: SupportsFloat, width: SupportsFloat):
    return __mul__(length, width)

measure_cave(12.0, 12.0)  # <-- works as before
measure_cave(12, 12)  # <-- passes now

What’s kinda cool about that approach is that this also works:

from numpy import float128
measure_cave(float128(12.0), float128(12.0))

Why the operator.__mul__, you ask? Great question! Thanks for asking. SupportsFloat is a Protocol that only requires that a thing define the __float__ method, not any operators. One alternative is:

@beartype
def measure_cave(length: SupportsFloat, width: SupportsFloat):
    return float(length) * float(width)

But I generally don’t like that, because conversions to native floats can be lossy. Be warned, however, that Protocols are devastatingly non-performant for runtime checks (if that’s what you find yourself doing). So much so that some of us have felt compelled to implement our own caching layer as a work-around. 🙄 If true support for arbitrary participants in the numeric tower is what you’re after, let me know if you’d like to discuss further, and I’ll carve out another place for that without further polluting your issue.

[leycec]

This is such a great issue. I’m so grateful to our crack commando squad of black-ops typing aficionados for repeatedly weighing in. They say what I cannot, for they are smart and I am tired. This is what happens when your eyes get encrusted with the detritus of sleep deprivation: you wait for @posita and @TeamSpen210 to pierce the darkness for you.

tl;dr

Yuppers. We’re prolly sticking with either:

# For those who worship before the graven alter of the official numeric tower:
from numbers import Number

# For those who desecrate idolatrous effigies of false abstract base classes:
from typing import Union
Number = Union[int, float, complex]

I am so, so sorry for what I have done to your codebase.

Why Are You So Horrible, @leycec?

Yes, @leycec is a Bad Enough Dude (BED).

With extreme apologies to AI King @KyleKing, ^{please don’t send the robot dogs} PEP 484, mypy, and (yuppers) Guido himself, @beartype would prefer to selectively pretend that something that was standardized wasn’t standardized. That is to say, we’re pretty sure Guido and those who approved PEP 484 got it wrong there.

As @posita himself unearthed from Guido’s inbox, ^{DropBox power!} Guido himself now acknowledges the ubiquitous world-straddling girth of PEP 3141. PEP 3141 came first and PEP 484 failed to deprecate PEP 3141; ergo, PEP 3141 takes precedence. Suck it, PEP 484.

Non-Euclidean Numeric Tower, We Invoke Thee!

So. Here we stand on the crumbling shores of a long-forgotten derelict civilization. The principal issues with PEP 3141 and the standard numbers module are (in increasing order of impotence):

• Everyone ignored it, because no one knew it even existed.
• Stack type checkers still ignore it, because they are dumb. ^{score 1: beartype}
• Protocols invite hidden O(n) costs, which is admittedly horrifying. Back before I became disillusioned by the veil of Maya gritty facade of reality, I was once pretty sure that ABCMeta efficiently cached (and thus effectively amortized) those costs. Having just reexamined the typing.py implementation in Python 3.10, I no longer know anything. Those costs might very well not be cached, in which case we can only collectively sigh and fling ourselves into the sea.

So. numbers had an advertising problem. But that wasn’t the fault of numbers; that was the fault of the humans surrounding numbers. Humanity, you have much to answer for.

So. numbers probably also has an efficiency problem. That absolutely is the fault of numbers and (more generally) the standard abc module underlying numbers. Remind me to never use ABCMeta in beartype. _{←oh gods we already did it 5 times}

What Could Possibly Go Wrong? Everything. Everything Could.

The principal issue with transparently treating float as int is that it’s wrong – especially in the numeric contexts that @beartype increasingly finds itself deployed to (e.g., finance, machine learning).

Floats aren’t integers. Integers aren’t floats. There exist a countably infinite number of integers with no precise floating-point representation; likewise, there exist an uncountably infinite number of floats with no corresponding integer representation.

Floats and integers are not losslessly coercible into one another is what I’m sayin’. Converting one into the other results in a loss of precision in both the average and worst case. This is why numeric frameworks like NumPy recently moved from implicit to explicit float <-> int coercions. Previously, NumPy implicitly coerced dtypes to and from float and int; now, NumPy requires you to explicitly permit NumPy to do that if you want NumPy to do that.

This is a good thing. Explicit is better than implicit, because we are all enlightened light beings here. You may now be thinking: “Go to Heck, @leycec! You go to Heck and sleep fitfully there.” Wait. I thought you were an enlightened light being!?!? I can only reply: “You may punch me in my fat gullet in five minutes if you remain fully unconvinced.” ^{would i lie}

If beartype implicitly accepts integers for all float type hints, then beartype prosumers will no longer be able to prevent their userbase from erroneously passing integers. Right? In 2021, most industrial-strength APIs really want to prevent their userbase from erroneously passing integers where floats are required. Currently, beartype supports this sensible constraint out-of-the-box at runtime – which is far more valuable than a static type checker supporting this sensible constraint out-of-the-box at static type-checking time.

If beartype implicitly accepts integers for all float type hints, then beartype prosumers will have to resort to beartype validators to force beartype to revert back to its prior sensible behaviour: e.g.,

from beartype.vale import Is
from typing import Annotated

# Type hint matching any non-integer float. Raise your voice in anger, userbase!
FloatNonint = Annotated[float, Is[
    lambda number: not isinstance(number, int)]]

Don’t Believe Me. Believe Those Who Believe Me.

alibi, a popular ML-splaining framework (it’s like mansplaining, only done by AI), wants to use beartype. This use is contigent on beartype behaving sanely. Currently, beartype behaves sanely by raising human-readable exceptions when @beartype-decorated callables annotated as accepting floats are passed integers. There are, like, a hundred of these test-time unit test failures, and stuff:

beartype.roar.BeartypeCallHintPepParamException: @beartyped
CounterfactualProto.__init__() parameter gamma="100" violates
type hint <class 'float'>, as "100" not instance of float.

Curiosity maimed the honey-addicted bear, so I noticed that and suggested they might globally replace float with numbers.Real in their type hints. Turns out they meant what they said and said what they meant. They actually did meant float, because all of their callables only accept floats; they do not accept integers. Now, they can trivially enforce this with righteous @beartype power:

Nice catch! I think this is an example where the parameter really should’ve been a float, beartype paying dividends already.

Are you gonna break alibi? I ain’t gonna break alibi. I like my kneecaps where they are – loosely attached to my legs.

Cost-Benefit Analysis

Now we’re into the hard stuff. Dust off those Grandma-era liquor cabinets, because we are going there.

These Are the Costs of an Irresponsible Life

Costs of @beartype not implicitly accepting integers for float type hints:

• RSI. Users type an additional 12 characters. Justifiable lawsuits ensue. @beartype collapses under a rising tide of unpaid legal fees and collective Internet outrage.

And… that’s about it. “But, wait! You calloused fool! What about mypy? Won’t someone think about the static type-checker?”, you are now venting. I can hear you over there, by the way.

Fine, fine. We don’t do what static type checkers do. But that’s permissible here, because static type checkers are (arguably) too permissive here. Mypy won’t emit errors or warnings for callables type-hinted with Union[float, int] instead of float, because the former is semantically equivalent to the latter from the static type-checking perspective.

So. The only cost remains RSI. You’ve got it. I’ve got it. We’ve all got it. What’s 12 characters more when your wrist has already been ground down into a thin bone spur of unadulterated pain?

These Are the Benefits of an Irresponsible Life

Benefits of @beartype not implicitly accepting integers for float type hints:

• Users catch nasty bugs. Does that sound like the principal benefit of type-checking to anyone else? Is it just me? Yes, passing an int where a float was expected is a bug in the general case, because: ^{…cue trippy acoustic guitar campfire sing-a-long}
  • Integers are not floats.
  • Floats are not integers.

These Are the Conclusions of the Starship Bearaprise

On the one hand, we’re giving everyone a permanent, crippling, debilitating physical injury that they already have. On the other hand, we’re doing our jobs.

And… We’re Goin’ Back to PEP 484

Anyone else notice the PEP 484 subsection we’re debating is super-weird? First, there’s this choice nugget:

There are some issues with these ABCs [i.e., numbers]…

There aren’t, actually. Okay, okay. O(n) runtime complexity is a problem – but not for most techbros. We’re the only ones who care about that. Anyone else notice how none of the supposed issues were actually enumerated or even mentioned? Guido throwin’ some baseless shade right there. Then there’s this even choicer nugget:

…but the built-in concrete numeric classes complex, float and int are ubiquitous (especially the latter two 😃.

OMG. There is not an ASCII emoji in my favourite PEP standard. There just isn’t. But… there is. Anyone else notice how this entire subsection reads more like casual email-speak than the quasi-formal standards-speak of the rest of the PEP? It’s almost like… no one actually even peer-reviewed or edited this subsection. Just sayin’.

We continue on our feeble death march with this frank admission:

Rather than requiring that users write import numbers and then use numbers.Float, etc…

Yes? And? Why aren’t we doing that? Because that’s what we should be doing.

…this PEP proposes a straightforward shortcut that is almost as effective:

Oh, bollocks. PEP subsection straight-up admitted it adopted a suboptimal, less effective solution than the previously standardized solution – just 'cause. No demonstrable justifications are given. Guido himself now acknowledges that numbers is preferable.

So, bollocks. We defy the established order, because we are beartype. We are preserving a distinction between float and int, because there is a distinction between float and int. The two are not transparently interchangeable and any attempt to treat them as such invites type safety violations and the immanent destruction of the supposedly incomplete Death Star.

A Pox on Beartype’s House

That said… let’s leave this open for all eternity.

There are extremely valid insights on bold sides of the opinion-littered back alley here. I invite them! Come! Berate me, excoriate my avatar, burn a miniature effigy of Mr. Nectar Palm (the @beartype mascot, yo) on your front lawn and then tweet us an animated GIF of the ensuing wreckage!

Do this in rembrance of this issue. Someday, even @leycec too may change his unsubstantiated worldview on QA.