python 3.8 raises warning in from beartype import beartype

Python 3.8 on Win10 raises warnings when using PYTHONDEVMODE=1:

C:\Program Files\Python39\lib\site-packages\beartype\_util\hint\pep\utilhintpeptest.py:367: BeartypeDecorHintPepDeprecatedWarning: PEP type hint typing.List[__main__.DbEntry] deprecated by PEP 585. To resolve this, globally replace this hint by the equivalent PEP 585 type hint (e.g., "typing.List[int]" by "list[int]"). See also:
    https://www.python.org/dev/peps/pep-0585
  warn(warning_message, BeartypeDecorHintPepDeprecatedWarning)
C:\Program Files\Python39\lib\site-packages\beartype\_util\hint\pep\utilhintpeptest.py:367: BeartypeDecorHintPepDeprecatedWarning: PEP type hint typing.Tuple[__main__.HintsHeader, typing.List[__main__.Hint]] deprecated by PEP 585. To resolve this, globally replace this hint by the equivalent PEP 585 type hint (e.g., "typing.List[int]" by "list[int]"). See also:
    https://www.python.org/dev/peps/pep-0585
  warn(warning_message, BeartypeDecorHintPepDeprecatedWarning)
C:\Program Files\Python39\lib\site-packages\beartype\_util\hint\pep\utilhintpeptest.py:367: BeartypeDecorHintPepDeprecatedWarning: PEP type hint typing.List[__main__.Hint] deprecated by PEP 585. To resolve this, globally replace this hint by the equivalent PEP 585 type hint (e.g., "typing.List[int]" by "list[int]"). See also:
    https://www.python.org/dev/peps/pep-0585
  warn(warning_message, BeartypeDecorHintPepDeprecatedWarning)

[leycec]

Thanks for the detailed response.

Thank you. No, really. You’ve taken precious time out of your Friday and Saturday for our humble project. I appreciate it. 👍

It looks like your Python3.9 is returning sys.version_info for 3.8 ??

You are, of course, correct. That was a copy-and-paste PEBKAC due to me being up so late that Friday night became Saturday morning. Gentoo’s Python3.9 is thankfully working as intended; here’s the actual output:

$ python3.8 -c 'import sys; print(sys.version_info)'
sys.version_info(major=3, minor=8, micro=7, releaselevel='final', serial=0)
$ python3.9 -c 'import sys; print(sys.version_info)'
sys.version_info(major=3, minor=9, micro=7, releaselevel='final', serial=0)

^phew

Does your Gentoo use the alternatives system or leave it up to you to name the right executable?

I’m so glad you asked, but you might not be in about five seconds. Gentoo has its own homegrown solution for installing multiple versions of the same underlying software. We call this stupefying technology slots.

Slots means that Python developers can easily install and update multiple versions of multiple Python interpreters and multiple versions of multiple Python packages for those multiple interpreters – all with a single command. Here is me installing mypy under CPython 3.7, 3.8, and 3.9 with a single invocation of emerge, the Gentoo package manager:

$ emerge mypy
These are the packages that would be merged:

Calculating dependencies                      ... done!         
[ebuild   R   ~] dev-python/mypy-0.800::gentoo  USE="-doc -test" PYTHON_TARGETS="python3_7 python3_8 python3_9" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No]

Gentoo thus installs, updates, and manages python3.7, python3.8, python3.9, and pypy3 commands and Python packages isolated to each of those commands for me on my behalf. It’s like pyenv + venv, only automated at the OS level without any manual intervention from me.

Command-line wizardry gets no hotter than that.

…back to reality.

I also use cygwin which creates a Posix environment on windows. It is intended to mimic Linux as closely as possible.

Yes! I love(d) Cygwin! I was reasonably active in the Babun community, until I realized that the Big List Of Dodgy Apps (BLODA) made Cywin and thus Babun effectively unusable for me. Then the Windows Subsystem for Linux (WSL) landed, which mostly solved everything despite being really slow, which admittedly is a blocker.

But I commend you for sticking with Cygwin. I certainly have fond memories of it – up until the End.

$ python3 -V
Python 3.8.7

> py -3.8 -V   &&   py -3.9 -V   &&   py -3.10 -V
Python 3.8.8
Python 3.9.2
Python 3.10.0a6

> py -3.8 -c "import sys; print(sys.version_info)"
sys.version_info(major=3, minor=8, micro=8, releaselevel='final', serial=0)

> py -3.9 -c "import sys; print(sys.version_info)"
sys.version_info(major=3, minor=9, micro=2, releaselevel='final', serial=0)

> py -3.10 -c "import sys; print(sys.version_info)"
sys.version_info(major=3, minor=10, micro=0, releaselevel='alpha', serial=6)

That looks sane! So far, so good.

I do not get the warnings from python 3.8.7, using -X dev or $ export PYTHONDEVMODE=1, so it looks like the warning behaviour has changed between 3.8.7 and 3.8.8. That is unexpected!

That looks distinctly less sane.

beartype contains conditional logic dependent on the current Python minor version (e.g., the 9 in 3.9.1), because type hinting syntax and semantics change across minor versions. beartype does not contain conditional logic dependent on the current Python patch version (e.g., the 1 in 3.9.1), because nothing is supposed to change across patch versions.

I’m also fairly sure that the behaviour of the Python Development Mode (PDM) does not change across patch versions (e.g., going from 3.8.7 to 3.8.8).

What you’re seeing is inexplicable. That troubles me. We’re probably to blame, because we usually are. Still… I’m having trouble thinking of a plausible explanation for that behaviour. Urgh!

Would doing something like List = list, etc instead of importing typing.List in your code where you can use list[], etc. for annotations work?

Oh. We seem to have confusion, which means I’m probably to blame. I’m awful at explanation, which is why I’m an engineer.

The deprecation warning you’re seeing isn’t about our code; it’s about your code. beartype is intentionally warning you that your code will break in four years, because most of the typing module is going away in four years. In four years, this is what is going to happen:

>>> from typing import List
ImportError: cannot import name 'List' from 'typing' (/usr/lib/python4.3/typing.py)

Or maybe you were asking whether substituting List = list in your own code would work? If so, the answer is sadly “Nope.” Why? Because that code would then break under Python 3.8, which doesn’t support PEP 585 type hints (e.g., list[str]). Static type checkers like mypy also won’t be fooled by those sorts of shenanigans. It’s best just to do the Right Thing™.

Now that you’re duly apprised and aware of PEP 585 deprecations, my advice as annotation guru would be to either:

1. Drop Python 3.8 support entirely and use PEP 585 type hints instead. PEP 585 type hints (e.g., list[str]) aren’t usable under Python 3.8, but they’re the future. Migrating from PEP 484 type hints (e.g., typing.List[str]) is non-trivial, because you have to stop importing from typing. It makes sense to pay that price now (when it doesn’t cost as much, because you haven’t written as much code) rather than later (when it will cost a lot more, because you’ll have written four years worth of code). Urgh!
2. Just hide the deprecation warning and pretend that everything’s okay. This is the easy option, which is nice. This also invites breakage in four years, which is bad. Hiding badness doesn’t make badness go away. But maybe you just need this to work now and don’t really care about the future? If so, that’s fine. Sometimes we just need code to work because someone is yelling at us. Here’s how to globally and permanently hide that warning in only three lines of Python:

from beartype.roar import BeartypeDecorHintPepDeprecatedWarning
import warnings
warnings.simplefilter(action='ignore', category=BeartypeDecorHintPepDeprecatedWarning)

<rant>

There are millions of lines of working Python code that will break when most of the typing module goes away in four years. Official Python developers will be breaking backward compatibility in a huge way… yet again. We all remember how well that Python 2 → 3 transition went, right?

But we can’t do anything about that except yell at official Python developers on their mailing lists and bug trackers. Of course, they won’t like that, so I advise not doing that. They may be objectively wrong here, but they still make the rules.

</rant>

The only issue for beartype here is that you’re seeing a deprecation warning under Python 3.8 that you shouldn’t. You should only be seeing that deprecation warning under Python 3.9 and 3.10. But… that’s hardly the worst thing to ever happen. We can’t replicate this and the fact that you see different behaviour between CPython 3.8.7 and 3.8.8 is a red flag of weirdness beyond our control.

BTW, does beartype check variable annotations? I did not see any mention in the doc.

I love fielding questions like this. Thanks for asking. 😄

The simple answer is: “Nope, but a related package called bearcall will let you do that soon!”

The complex answer is that variable annotations are out-of-scope for beartype itself. Variable annotations were standardized by PEP 526, which beartype explicitly does not support. beartype operates at the coarse-grained level of functions and methods rather than the fine-grained level of individual statements in functions and methods.

Instead, we’re making a new Python project later this year called bearcall. Unlike beartype, bearcall will operate at the fine-grained level of individual statements in functions and methods – including assignment statements with variable annotations. Specifically, bearcall will let you type-check any arbitrary object at any arbitrary time against any PEP-compliant type hint by passing that object and hint to a simple bearcall() validation function: e.g.,

from bearcall import bearcall

# List of 1,000 integers both type-checked at runtime *AND*
# type-checkable by static type checkers like mypy.
THOUSAND_INTS: list[int] = bearcall(list(range(1000)), list[int])

The bearcall() API intentionally resembles the isinstance() builtin. Anything you can pass to isinstance(), you can also pass to bearcall(). The reverse is naturally not true, because bearcall() will also check arbitrary objects against arbitrarily complex type hints in O(1) time. bearcall will be internally implemented with beartype – giving you the full power, efficiency, and expressiveness of beartype anywhere.

There’s a bit of a DRY violation above, because we repeat the list[int] type hint twice. That’s unavoidable within functions, because CPython doesn’t actually store local variable annotations anywhere; they’re just stripped and ignored at runtime.

Module and class variables are a different story. CPython does store those somewhere (i.e., in an __annotations__ dunder dictionary), so bearcall could also offer a bearcall_nonlocal() validation function accepting either a module or class variable and automatically looking up that variable’s type hint for you: e.g.,

from bearcall import bearcall_nonlocal

# List of 1,000 integers both type-checked at runtime *AND*
# type-checkable by static type checkers like mypy.
THOUSAND_INTS: list[int] = list(range(1000))
bearcall_nonlocal(THOUSAND_INTS)

Is that something you’d be interested in? If so, I’d be happy to reprioritize. I was holding off on bearcall until I’d released beartype 1.0.0, which wouldn’t be until the end of this year. If there’s interest, I’d consider pushing bearcall forward into the earlier part of this year. Just let me know, eh?

Thanks again for all the interest, Doug! I sure hope you’re having a warmer weekend over there than we are here in Ontario. Brrrrrrrr. ❄️

[leycec]

Wow. I just checked out pywikitree, your impressive Python-based WikiTree API. That’s just wizardly! Tracking my wife’s genealogy would be equally fascinating and depressing, because her mother’s family fled Lithuania during the German-Soviet invasion of World War II on horseback over sickeningly tall mountain ranges that I’d be lucky to even stagger up today. 🗻