question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Some numpy broadcasting operations fail in numba 0.51

See original GitHub issue

Minimal code example:

import numpy as np
from numba import njit

@njit(parallel=True)
def use_numba():
    temp = np.ones((3, 5))
    temp[1:] *= (2*np.ones(5))
    return temp                                                                                                                                                                                                                       print(use_numba())

(I’m using Python 3.8 but guess this is irrelevant) in numba 0.49.1 this works fine and gives:

[[1. 1. 1. 1. 1.]
 [2. 2. 2. 2. 2.]
 [2. 2. 2. 2. 2.]]

but in numba 0.51.2 it gives an error, I think it is a broadcasting issue between temp which is 2D and the other factor which is 1D:

TypingError: Failed in nopython mode pipeline (step: nopython mode backend) 
Failed in nopython mode pipeline (step: nopython frontend) 
No implementation of function Function(<built-in function getitem>) found for signature: 
  
 >>> getitem(array(float64, 1d, C), UniTuple(uint64 x 2)) 
  
There are 22 candidate implementations: 
  - Of which 20 did not match due to: 
  Overload of function 'getitem': File: <numerous>: Line N/A. 
    With argument(s): '(array(float64, 1d, C), UniTuple(uint64 x 2))': 
   No match. 
  - Of which 2 did not match due to: 
  Overload in function 'GetItemBuffer.generic': File: numba/core/typing/arraydecl.py: Line 162. 
    With argument(s): '(array(float64, 1d, C), UniTuple(uint64 x 2))': 
   Rejected as the implementation raised a specific error: 
     TypeError: cannot index array(float64, 1d, C) with 2 indices: UniTuple(uint64 x 2) 
  raised from /home/zonca/anaconda/envs/temp/lib/python3.8/site-packages/numba/core/typing/arraydecl.py:84 
 
During: typing of intrinsic-call at /extrahome/software/pysm/minimal.py (7) 
 
File "minimal.py", line 7: 
def use_numba(): 
    <source elided> 
    temp = np.ones((3, 5)) 
    temp[1:] *= (2*np.ones(5)) 
    ^ 
 
During: lowering "id=3[LoopNest(index_variable = parfor_index.29, range = (0, $22binary_subscr.11_size0.1, 1)), LoopNest(index_variable = parfor_index.30, range = (0, $const_5, 1))]{4: <ir.Block at /extrahome/software/pysm/minimal.py (7)>}Var($parfor_index_tuple_var.31, minimal.py:7)" at /extrahome/software/pysm/minimal.py (7) 

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
stuartarchibaldcommented, Sep 9, 2020

Thanks @zonca, I’ve found it:

$ git bisect start
$ git bisect bad HEAD
$ git bisect good 0.51.0dev0
Bisecting: 321 revisions left to test after this (roughly 8 steps)
[4c57a57eccbff75b834f39ea5792e67cd58c3abc] Merge pull request #6013 from yumasheta/update-cuda-kernel-docs
$ git bisect run python issue6228.py
running python issue6228.py
Bisecting: 162 revisions left to test after this (roughly 7 steps)
[bed4d046e10e2e74012330e840be3982ca714c21] Merge pull request #6058 from sklam/fix/overspecialize_option
running python issue6228.py
Bisecting: 78 revisions left to test after this (roughly 6 steps)
[6c01e9ab74efb84094c8a443a8635b696f5b2d88] Merge pull request #6052 from gmarkall/grm-fix-atomic-tests-dtypes
running python issue6228.py
Bisecting: 39 revisions left to test after this (roughly 5 steps)
[b99ca565c910dd1ac760947fc19822f7401f2586] Add support for CUDA stream callbacks and asyncio awaitable streams
running python issue6228.py
Bisecting: 19 revisions left to test after this (roughly 4 steps)
[e686e52bbbcfe80b721e1512011cfe85b0e73b17] Merge pull request #5846 from gmarkall/grm-debug-fixes
running python issue6228.py
Bisecting: 9 revisions left to test after this (roughly 3 steps)
[76fc4643c172df4f9b5812530e976562fbf109a6] Attempt to fix test_const_record_optimization on Windows / CUDA 10
running python issue6228.py
Bisecting: 4 revisions left to test after this (roughly 2 steps)
[d8a272eeeb0420c68d9594b8b1b3e633ea4ac098] Move the conversion of inplace binops after the prange pass so that reductions in prange are not interpreted as inplace binops.  Unify the calltypes of the binop creates in the inplace_binop parfor body.
running python issue6228.py
Bisecting: 2 revisions left to test after this (roughly 1 step)
[d72383667c90bbf7d4b9663288eb18ceff83cca4] Add assignment after inplace_binop parfor to assign target of inplace binop to lhs of the binop.
running python issue6228.py
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[d7721cbb3b4777f818afc79d43bc99ff96be9af4] Add parfor support for inplace_binop on arrays.
running python issue6228.py
d7721cbb3b4777f818afc79d43bc99ff96be9af4 is the first bad commit
commit d7721cbb3b4777f818afc79d43bc99ff96be9af4
Author: <redacted>
Date:   Thu Jun 11 09:17:05 2020 -0700

    Add parfor support for inplace_binop on arrays.

 numba/core/cpu_options.py |   2 +
 numba/parfors/parfor.py   | 123 ++++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 120 insertions(+), 5 deletions(-)
bisect run success

the new support for inplace binops on arrays caused this regression.

CC @DrTodd13

0reactions
zoncacommented, Sep 10, 2020

@DrTodd13 I also confirm that disabling inplace_binops works:

import numpy as np
from numba import njit

@njit(parallel={"inplace_binop":False})
def use_numba():
    temp = np.ones((3, 5))
    temp[1:] *= (2*np.ones(5))
    return temp

print(use_numba())

works fine.

Read more comments on GitHub >

github_iconTop Results From Across the Web

5 minute guide to Numba
A ~5 minute guide to Numba¶. Numba is a just-in-time compiler for Python that works best on code that uses NumPy arrays and...
Read more >
Troubleshooting and tips - Numba documentation
The most common reason for slowness of a compiled JIT function is that compiling in nopython mode has failed and the Numba compiler...
Read more >
Faster Python with NumPy broadcasting and Numba
Array Broadcasting's pros: you only write Python code; it is very fast for any array size; there is no compilation overhead; you don't...
Read more >
importing numba module shows ImportError - Stack Overflow
from numba import jit import numpy as np x = np.arange(100).reshape(10, 10) @jit(nopython=True) # Set "nopython" mode for best performance, ...
Read more >
Dynamic Programming on the GPU via JAX - QuantEcon - Notes
JAX prefers vectorized operations, meaning that loops need to be replaced by operations on arrays. We use some NumPy broadcasting tricks to ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found