question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DFS-based backedge algorithm can be slow for pathological cases

See original GitHub issue

cc @sklam @quasiben @philippjfr @AjayThorve

See https://github.com/holoviz/datashader/issues/956

No Numba-only reproducer as yet, but the DFS-based algorithm increases the runtime of the reproducer in the Datashader issue from 10 seconds to 34 minutes. Going back to the old algorithm with:

diff --git a/numba/core/controlflow.py b/numba/core/controlflow.py
index f68572031..f37e9ae3d 100644
--- a/numba/core/controlflow.py
+++ b/numba/core/controlflow.py
@@ -572,40 +572,14 @@ class CFGraph(object):
         Find back edges.  An edge (src, dest) is a back edge if and
         only if *dest* dominates *src*.
         """
-
-        # Uses a simple DFS to find back-edges.
-        # The new algorithm is faster than the the previous dominator based
-        # algorithm.
         back_edges = set()
-        # stack: keeps track of the traversal path
-        stack = []
-        # succs_state: keep track of unvisited successors of a node
-        succs_state = {}
-        entry_point = self.entry_point()
-
-        def push_state(node):
-            stack.append(node)
-            succs_state[node] = [dest for dest in self._succs[node]]
-
-        push_state(entry_point)

-        while stack:
-            tos = stack[-1]
-            tos_succs = succs_state[tos]
-            # Are there successors not checked?
-            if tos_succs:
-                # Check the next successor
-                cur_node = tos_succs.pop()
-                # Is it in our traversal path?
-                if cur_node in stack:
-                    # Yes, it's a backedge
-                    back_edges.add((tos, cur_node))
-                else:
-                    # Push
-                    push_state(cur_node)
-            else:
-                # Checked all successors. Pop
-                stack.pop()
+        for src, succs in self._succs.items():
+            back = self._doms[src] & succs
+            # In CPython bytecode, at most one back edge can flow from a
+            # given block.
+            assert len(back) <= 1
+            back_edges.update((src, dest) for dest in back)

         return back_edges

Brings the runtime back down again:

$ time python repro.py
x_range: (1285891200000000000, 1291891140000000000) y_range: (-104.8094774985483, 100.09816883332905)

real    0m9.732s
user    0m9.601s
sys     0m0.113s

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
gmarkallcommented, Aug 29, 2020

Seems that #6187 could be a potential fix, but we could still do with a Numba-only reproducer.

0reactions
jakirkhamcommented, Sep 3, 2020

Thanks all! 😄

Read more comments on GitHub >

github_iconTop Results From Across the Web

Tree, Back, Edge and Cross Edges in DFS of Graph
Tree Edge: It is an edge which is present in the tree obtained after applying DFS on the graph. All the Green edges...
Read more >
Assignment 5 - Mark Dolan Programming - Google Sites
Classify each edge as a tree edge or back edge, and give the pre and post ... Run the DFS-based topological ordering algorithm...
Read more >
DFS | Tree Edge, Back Edge, Forward Edge, Cross Edge
In this video we see the classification of edges in DFS of a graph.In Directed Graph:- Tree Edge,- Forward Edge,- Back Edge,- Cross...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found