Use next in iterator objects

Python 3.7 vt-py 0.5.4

import vt

client = vt.Client(VT_API_KEY)
query = 'entity:domain p:0 urls_max_detections:10+'
attacked_legit_domains = client.iterator('/intelligence/search', params={'query': query}, limit=1)

legit_domain = next(attacked_legit_domains)

Traceback:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-18-806f21dacbfe> in <module>()
      2 attacked_legit_domains = client.iterator('/intelligence/search', params={'query': query}, limit=1)
      3 
----> 4 legit_domain = next(attacked_legit_domains)

TypeError: 'Iterator' object is not an iterator

[mgmacias95]

Hello @carlosjcabello,

Reading PEP 525, I understand they wanted to iterate over generators using async for without implementing an iterator obejct. Compare the following outputs (Python 3.5 vs Python 3.9):

Python 3.5.10 (default, Sep 10 2020, 18:30:47) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> async def my_generator():
...     yield 1
...     yield 2
... 
  File "<stdin>", line 2
SyntaxError: 'yield' inside async function
>>> def my_generator():
...     yield 1
...     yield 2
... 
>>> async def async_iteration():
...     async for i in my_generator():
...             print(i)
... 
>>> evl = asyncio.get_event_loop()
>>> evl.run_until_complete(async_iteration())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/asyncio/base_events.py", line 467, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.5/asyncio/futures.py", line 294, in result
    raise self._exception
  File "/usr/local/lib/python3.5/asyncio/tasks.py", line 240, in _step
    result = coro.send(None)
  File "<stdin>", line 2, in async_iteration
TypeError: 'async for' requires an object with __aiter__ method, got generator
>>> 

Python 3.9.4 (default, Apr  5 2021, 01:50:46) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> 
>>> async def my_generator():
...     yield 1
...     yield 2
... 
>>> async def async_iteration():
...     async for i in my_generator():
...             print(i)
... 
>>> asyncio.run(async_iteration())
1
2

That said, the code is actually implementing an iterator per sé, so the implementation you did is more correct in terms of the concept being implemented than the implementation made at #12.

Regards, Marta

[carlosjcabello]

Hi Marta,

Thanks for the additional details.

I created the draft PR https://github.com/VirusTotal/vt-py/pull/68 based on https://github.com/VirusTotal/vt-py/issues/24#issuecomment-893567216. On it, __iter__ and __aiter__ simply return the instance self and all the iteration logic is moved to __next__ and __anext__ methods.

Elaborating more on my previous comment, what I mean is that there was a similar implementation before https://github.com/VirusTotal/vt-py/pull/12. That PR changed the iterator implementation to be based on generators. As @chexca and @plusvic mentioned, it made the code simpler. But it had as a consequence that ‘Iterator’ class instances were not iterators objects. Now we are changing the implementation again, moving away from generators.

Just wanted to know if there was a consensus at VT on this. That is, that the proposed solution is preferred over the generator-based implementation.