Debugger silently calls __iter__() and next()

Try to debug this code:

import os
class MyIterator(list):
	def __iter__(self): return self
	def next(self): os._exit(1)
a = MyIterator()
print(a)   # put a breakpoint here

You will find that the debugger will quit before you even attempt to do anything.

This should not happen. If the debugger insists on iterating through members of every subclass of list silently, then it should really call list.__iter__(instance) rather than iter(instance). Otherwise it breaks debugging on the entire codebase, since the iterator is silently consumed and hence the program no longer behaves correctly when stepping through.

Note that similar problems probably exist for other types, not just list. More generally, I think the debugging experience of subclasses of basic types needs improvement; it is currently not robust.

(This bug probably touches on similar aspects of the debugger as #3505.)

[zooba]

As we discussed on the other bug, “improving” debugging of subclasses really just means disabling it. We need to balance that against user expectations, which is that objects implementing a certain protocol will adhere to that protocol - by convention, iteration should not terminate the process, and in particular iteration of an iterable should not have observable side effects. Our debugger breaking here is a good indication that anyone using that code will likely also be broken.

Basically, we don’t want to cripple most users’ experiences just in case someone broke the rules and expects their broken code to still work correctly.

[int19h]

The check for iter(self) is self follows the iterator protocol - per PEP 234:

A class that wants to be an iterator should implement two methods: a next() method that behaves as described above, and an iter() method that returns self.

I agree that it’s a hack - it’s not really meant to detect iterators - but it’s the best one that we can come up with, since there’s no standard way to do so with Python (although it would be nice to have a way to ask any object if iterating it is one-off or not, and whether it’s expensive or not).

That said, looking at the code more closely, I’m not sure if this is actually the code path that leads to your problem. It appears that when we’re reporting children (this is after retrieving them in enum_child_locally), we’re actually calling len() on each of those values:

def report_children(execution_id, children):
    children = [(name, expression, flags, safe_repr(result), safe_hex_repr(result), type(result), type(result).__name__, get_object_len(result)) for name, expression, result, flags in children]

where get_object_len is:

def get_object_len(obj):
    try:
        return len(obj)
    except:
        return None

and ditto in report_execution_result. And when used on an iterator, len will iterate it. The same logic should be extended there.