You gotta keep ’em iterated

(With apologies to The Offspring.)

So, I’m…lazy. I work really hard at it. (I’m likely not the only one.) Once upon a time, when I was new to Python typing, spending hours reading about generators, iterables, iterators, sequences, etc., instead of minutes coding up what I needed to … . Wait. You know what? For once, I’m not going to craft another impromptu episode of Time for a Trivial Moral That Didn’t Benefit a Lick by Being Buried at the End of a Long, Rambling Story but Instead Could Have and Should Have Led, Because at Least the Audience Would Have Been Awake at the Time™. (Title capitalization verified, so you know it’s authentic.)

Anyhoo, this:

A generator can be annotated by the generic type Generator[YieldType, SendType, ReturnType]. … [¶] … [¶] Alternatively, annotate your generator as having a return type of either Iterable[YieldType] or Iterator[YieldType] … .

Now you and I both know that taking certain parts of the standard library documentation at face value can lead to great sadness, but I think the quoted statement is legit in spirit, at least insofar as iterators do appear to be subsets of generators.

[sweeping gesture to divert attention away from the fact that immediately after conceding that Python’s docs were not necessarily trustworthy, evidence presented in support of why they should be trusted in this case came from those very same docs]

Over my career, I would conservatively estimate that typing Iterator[Foo] instead of Generator[Foo, None, None] has saved me dozens of precious seconds that I would have never been able to squander on one fifth of one YouTube video recommendation that briefly compelled me to sit—motionless—until its banal end, but of which I now possess no memory.

My point is that my code is littered with stuff like this:

def ima_generator_fool(…) -> Iterator[…]:
    …
    yield …
    …

Sadly, with 0.9.0, I must now account for my sloth:

# test_case.py
from __future__ import annotations
from collections.abc import Generator, Iterator, Sequence
from beartype import __version__ as beartype_version
from beartype import beartype
print(beartype_version)

@beartype
def silly_rator1(seq: Sequence) -> Generator:  # <-- Generator works ...
    yield from seq

@beartype
def silly_rator2(seq: Sequence) -> Iterator:  # <-- ... but Iterator fails
    yield from seq

% PYTHONWARNINGS=ignore python test_case.py
0.9.0
Traceback (most recent call last):
  File "/Users/matt/Documents/dev/dycelib/test_case.py", line 9, in <module>
    def silly_rator1(seq: Sequence) -> Generator:  # <-- Generator works ...
  File "/Users/matt/.acct/.virtualenvs/dycelib/lib/python3.9/site-packages/beartype/_decor/main.py", line 159, in beartype
    func_wrapper_code = generate_code(func_data)
  File "/Users/matt/.acct/.virtualenvs/dycelib/lib/python3.9/site-packages/beartype/_decor/_code/codemain.py", line 170, in generate_code
    code_check_return = _code_check_return(data)
  File "/Users/matt/.acct/.virtualenvs/dycelib/lib/python3.9/site-packages/beartype/_decor/_code/codemain.py", line 450, in _code_check_return
    hint = reduce_hint_pep484585_func_return(func)
  File "/Users/matt/.acct/.virtualenvs/dycelib/lib/python3.9/site-packages/beartype/_util/hint/pep/proposal/pep484585/utilpep484585func.py", line 129, in reduce_hint_pep484585_func_return
    _die_of_hint_return_invalid(
  File "/Users/matt/.acct/.virtualenvs/dycelib/lib/python3.9/site-packages/beartype/_util/hint/pep/proposal/pep484585/utilpep484585func.py", line 182, in _die_of_hint_return_invalid
    raise exception_cls(
beartype.roar.BeartypeDecorHintPep484585Exception: @beartyped generator silly_rator1() return type hint <class 'collections.abc.Generator'> contextually invalid (i.e., expected either collections.abc.Generator[...] or typing.Generator[...] type hint)..

FYI, Importing from typing instead of collections.abc has the same effect. This worked in 0.8.1:

% PYTHONWARNINGS=ignore python test_case.py
0.8.1

Also, Mypy doesn’t complain, and I vaguely recall a great man once saying, “What Mypy does, [beartype] do.”

[leycec]

this. cannot. be.

Lies! Filthy lies! …very well. I’ve now imbibed a generous fistful of quasi-legal Canadian herbal supplements and should be awash in a semblance of pharmaceutical enlightenment for the next five minutes. Let’s see how far we can run with that.

Awash in a Sea of Hearsay, I Say

So. On the one hand, I’m dead ^…gulp certain that whoever documented typing.Generator just casually invented the following sus af sentence without reference to any existing PEP standard:

Alternatively, annotate your generator as having a return type of either Iterable[YieldType] or Iterator[YieldType]:

I’m equally dead ^{…gulp, gulp} certain that whoever wasn’t Guido, because Guido rarely authors his own docos, because ain’t nobody whose middle name means “A large covered wagon for moving furniture, etc., also for conveying wild beasts, etc., for exhibition.” (thanks, Webster 1913 edition) got time for docos.

Notably, PEP 484’s “Annotating generator functions and coroutines” subsection fails to define this equivalence despite defining a similar equivalence for coroutines (i.e., Coroutine[None, None, {returns}] is safely reducible to just {returns}). No other PEP even dares to comment on this super-dodgy subject.

This is the Python equivalent of Russian chatbot agitprop.

Pour One Out for the Propagandists

On the other hand, I for one am all for Russian chatbot agitprop. AI Daddy always knows best, I’ve always said! ^{nervous tinkling laughter} Thankfully, this is no exception or I’d now be looking over my shoulder in trepidation.

Generator[{yields}, None, None] should be safely reducible to both Iterable[{yields}] and Iterator[{yields}]. It’s not just the common case; it’s really the only case. I’m even deader ^{…gulp, gulp, gulp} certain no one still living has authored a real-world working generator either accepting something via send() or returning something via return. No one does that or my middle name isn’t Bearmichaelson. Moreover, as Matt notes, generators explicitly satisfy both the Iterator and thus Iterable APIs.

And then there’s everything Spen said, who retains a perfect batting record of 10/10 wiseguy comments. Anyone else notice that? I can’t be the only one. Dude’s always right. How does that even happen? No living, breathing human is that infallible. Unless… oh my gods no–

Do It for Matty and Spenny, Man

So. It’s happening! We are unflipping the table. We are implementing this equivalence. We are formally apologizing before a tight-lipped delegation of the U.N. Security Council for abusing @posita’s dwindling YouTube agitprop consumption time. Game of Thrones Season Eight ain’t gonna watch itself, after all. </shudders_spasmodically>

[posita]

I grin, nod, and ask, “Why?” Then someone either punches me, fires me, or pretends I just said posed an existentialist conundrum in Kiswahili.

You and I must have both had chicken pox the week they taught ~obedience~ market consumption for great good in ~training camp~ school. If I had a nickel every time something like what you describe happened to me, I’d probably have a fist full of nickels.