question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Optimise range() iteration despite type casts on the step value

See original GitHub issue

Given the code below, all of the functions compile except for mysum2.

cdef double mysum0(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>len(a)):
        r += a[i]
    return r


cdef double mysum1(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a)):
        r += a[i]
    return r


cdef double mysum2(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a), <size_t>1):
        r += a[i]
    return r

The errors generated by Cython for mysum2 are as follows.

Error compiling Cython file:
------------------------------------------------------------
...


cdef double mysum2(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a), <size_t>1):
            ^
------------------------------------------------------------

test_range.pyx:20:13: Coercion from Python not allowed without the GIL

Error compiling Cython file:
------------------------------------------------------------
...


cdef double mysum2(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a), <size_t>1):
            ^
------------------------------------------------------------

test_range.pyx:20:13: Iterating over Python object not allowed without gil

Error compiling Cython file:
------------------------------------------------------------
...


cdef double mysum2(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a), <size_t>1):
                 ^
------------------------------------------------------------

test_range.pyx:20:18: Calling gil-requiring function not allowed without gil

Error compiling Cython file:
------------------------------------------------------------
...


cdef double mysum2(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a), <size_t>1):
                 ^
------------------------------------------------------------

test_range.pyx:20:18: Constructing Python tuple not allowed without gil

Error compiling Cython file:
------------------------------------------------------------
...


cdef double mysum2(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a), <size_t>1):
                  ^
------------------------------------------------------------

test_range.pyx:20:19: Converting to Python object not allowed without gil

Error compiling Cython file:
------------------------------------------------------------
...


cdef double mysum2(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a), <size_t>1):
                             ^
------------------------------------------------------------

test_range.pyx:20:30: Converting to Python object not allowed without gil

Error compiling Cython file:
------------------------------------------------------------
...


cdef double mysum2(double[:] a) nogil:
    cdef size_t i
    cdef double r = 0
    for i in range(<size_t>0, <size_t>len(a), <size_t>1):
                                             ^
------------------------------------------------------------

test_range.pyx:20:46: Converting to Python object not allowed without gil

Details about the environment used below. This was run on my Mac, but expect the error to occur independently of the OS used.

name: cython
channels:
  - conda-forge
  - defaults
dependencies:
  - appdirs=1.4.3=py_1
  - appnope=0.1.0=py36_0
  - asn1crypto=0.24.0=py_1
  - attrs=18.1.0=py_1
  - automat=0.7.0=py36_0
  - backcall=0.1.0=py_0
  - blas=1.1=openblas
  - bleach=2.1.3=py_0
  - ca-certificates=2018.4.16=0
  - certifi=2018.4.16=py36_0
  - cffi=1.11.5=py36_0
  - constantly=15.1.0=py_0
  - cryptography=2.3=py36hdffb7b8_0
  - cryptography-vectors=2.3=py36_0
  - cython=0.28.4=py36hfc679d8_0
  - decorator=4.3.0=py_0
  - entrypoints=0.2.3=py36_1
  - html5lib=1.0.1=py_0
  - hyperlink=17.3.1=py_0
  - idna=2.7=py36_2
  - incremental=17.5.0=py_0
  - ipykernel=4.8.2=py36_0
  - ipython=6.4.0=py36_0
  - ipython_genutils=0.2.0=py_1
  - jedi=0.12.0=py36_0
  - jinja2=2.10=py_1
  - jsonschema=2.6.0=py36_1
  - jupyter_client=5.2.3=py_1
  - jupyter_core=4.4.0=py_0
  - libffi=3.2.1=3
  - libgfortran=3.0.0=0
  - libsodium=1.0.16=0
  - line_profiler=2.1.2=py36h470a237_1
  - markupsafe=1.0=py36_0
  - mistune=0.8.3=py36_1
  - nbconvert=5.3.1=py_1
  - nbformat=4.4.0=py_1
  - ncurses=6.1=0
  - notebook=5.6.0=py36_0
  - numpy=1.14.5=py36_blas_openblashd3ea46f_201
  - openblas=0.2.20=8
  - openssl=1.0.2o=0
  - pandoc=2.2.2=hde52d81_1
  - pandocfilters=1.4.2=py_1
  - parso=0.3.0=py_0
  - pexpect=4.6.0=py36_0
  - pickleshare=0.7.4=py36_0
  - prometheus_client=0.3.0=py_0
  - prompt_toolkit=1.0.15=py36_0
  - ptyprocess=0.6.0=py36_0
  - pyasn1=0.4.3=py_0
  - pyasn1-modules=0.2.1=py_0
  - pycparser=2.18=py_1
  - pygments=2.2.0=py_1
  - pyhamcrest=1.9.0=py_2
  - pyopenssl=18.0.0=py36_0
  - python=3.6.5=1
  - python-dateutil=2.7.3=py_0
  - pyzmq=17.1.0=py36hae99301_0
  - readline=7.0=haf1bffa_1
  - scipy=1.1.0=py36_blas_openblashd3ea46f_201
  - send2trash=1.5.0=py_0
  - service_identity=17.0.0=py_0
  - setuptools=40.0.0=py36_0
  - simplegeneric=0.8.1=py_1
  - six=1.11.0=py36_1
  - sqlite=3.20.1=0
  - terminado=0.8.1=py36_0
  - testpath=0.3.1=py36_0
  - tk=8.6.8=0
  - tornado=5.1=py36_0
  - traitlets=4.3.2=py36_0
  - wcwidth=0.1.7=py_1
  - webencodings=0.5.1=py36_0
  - xz=5.2.3=0
  - zeromq=4.2.5=hfc679d8_4
  - zlib=1.2.11=h470a237_3
  - zope.interface=4.5.0=py36h470a237_0
  - twisted=17.5.0=py36_0

Issue Analytics

  • State:open
  • Created 5 years ago
  • Comments:20 (16 by maintainers)

github_iconTop GitHub Comments

1reaction
jimy-byerleycommented, Feb 12, 2022

That’s what I have done so far. 😐 There was nothing else at the time

1reaction
scodercommented, Feb 10, 2022

Note that completing PR #368 would probably get us most of the benefit – this issue is then only a small optimisation on top. See #1106.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Python range() Function Explained with Examples - PYnative
Python range() returns the sequence of numbers starting from a given start integer to a stop integer, which we can iterate using a...
Read more >
Optimize Options (Using the GNU Compiler Collection (GCC))
This option tells the loop optimizer to use language constraints to derive bounds for the number of iterations of a loop. This assumes...
Read more >
PythonSpeed/PerformanceTips - Python Wiki
Overview: Optimize what needs optimizing. You can only know what makes your program slow after first getting the program to give correct results ......
Read more >
How to iterate over a list in chunks - python - Stack Overflow
from itertools import islice list_ = [i for i in range(10, 100)] def chunker(it, size): iterator = iter(it) while chunk := list(islice(iterator, size)): ......
Read more >
If you have slow loops in Python, you can fix it…until you can't
Let's take a computational problem as an example, write some code, ... In other words, you are to maximize the total value of...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found