question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Replace `all(list comprehension)` with `all(generator)` to lazily evaluate elements

See original GitHub issue

Replace:

all([i == 0 for in range(3)])
any([i == 0 for in range(3)])

with:

all(i == 0 for in range(3))
any(i == 0 for in range(3))

Why do we need this change?

https://eklitzke.org/generator-comprehensions-and-using-any-and-all-in-python

Instructions:

  1. Copy & save this script.
import os
import sys
import ast
from pathlib import Path
from typing import Iterable, List


def iter_python_scripts(root: str) -> Iterable[Path]:
    for p in Path(root).rglob("*"):
        if p.name.endswith(".py"):
            yield p


def read_file(path: Path) -> str:
    return path.read_text()


def create_position_link(path: str, lineno: int, col_offset: int) -> str:
    return f"{path}:{lineno}:{col_offset}"


def is_list_comp(n: ast.AST):
    return isinstance(n, ast.ListComp)


class Visitor(ast.NodeVisitor):
    def __init__(self) -> None:
        super().__init__()
        self.nodes: List[ast.Call] = []

    def visit_Call(self, node: ast.Call) -> None:
        if (
            isinstance(node.func, ast.Name)
            and node.func.id in ["any", "all"]
            and any(map(is_list_comp, node.args))
        ):
            self.nodes.append(node)
        self.generic_visit(node)


def main() -> None:
    for d in ["mlflow", "tests"]:
        for path in iter_python_scripts(d):
            src = read_file(path)
            root = ast.parse(src)
            visitor = Visitor()
            visitor.visit(root)

            if len(visitor.nodes) == 0:
                continue

            print(
                "\n".join(create_position_link(path, n.lineno, n.col_offset) for n in visitor.nodes)
            )


if __name__ == "__main__":
    main()

  1. Run the script in the repository root to list the lines containing all(list comprehension) or any(list comprehension):
% python a.py
tests/types/test_schema.py:231:11
tests/sklearn/test_sklearn_model_export.py:532:11
tests/sklearn/test_sklearn_model_export.py:562:15
tests/tensorflow/test_tensorflow2_autolog.py:764:11
tests/autologging/test_autologging_behaviors_unit.py:184:11
tests/autologging/test_autologging_behaviors_unit.py:190:11
tests/autologging/test_autologging_behaviors_integration.py:284:11
tests/autologging/test_autologging_behaviors_integration.py:302:11
tests/autologging/test_autologging_safety_unit.py:635:11
tests/autologging/test_autologging_safety_unit.py:636:11
tests/autologging/test_autologging_safety_unit.py:637:11
tests/tracking/test_tracking.py:285:11
tests/pytorch/test_pytorch_model_export.py:564:11
...
  1. Fix the listed lines.
  2. Run the script again and make sure it prints out nothing.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
nchittelacommented, Apr 8, 2022

@harupy i’ve just submitted a PR #5640. Please let me know if you have any feedback 😃

Thank you @akrabdev for the opportunity!

1reaction
harupycommented, Apr 8, 2022

Thanks @akrabdev for giving @nchittela an oppotunity to contribute to OSS 😃

Read more comments on GitHub >

github_iconTop Results From Across the Web

Generator expressions vs. list comprehensions - Stack Overflow
These functions are designed to stop iterating when the answer is known, but a list comprehension must evaluate every element before the function...
Read more >
When to Use a List Comprehension in Python
Your set comprehension outputs all the unique vowels it found in quote . Unlike lists, sets don't guarantee that items will be saved...
Read more >
List Comprehensions in Python and Generator Expressions
Lists are mutable in Python. This means you can replace, add or remove elements. How to create lists. There are 2 common ways...
Read more >
Python Language Tutorial => List Comprehensions
All results are returned at once in the new list. Generator expressions are evaluated lazily, but list comprehensions evaluate the entire iterator ...
Read more >
Does Python still need the map() function? | by Marcin Kozak
You can read about all these things in other articles published on Medium. ... the map() version or the generator expression (or a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found