question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow getting pointer (address) to Numba's int/float variables

See original GitHub issue

Feature Request


For example I have some cffi or ctypes function that has a pointer int64_t * as one of its argument. I need to call this function passing pointer (address) of single-value np.int64 variable. I found a way how to get and pass a pointer to any element of Numpy array, using .ctypes property. But didn’t find any way of passing a pointer to int64 (np.int64 or numba.int64). Example code:

import numba, numpy as np

@numba.njit(cache = True)
def f():
    a = np.zeros((10,), dtype = np.int64)
    aptr = a[1:3].ctypes
    b = np.int64
    bptr = b.ctypes
    c = numba.int64
    cptr = c.ctypes
    
f()

In code above Numpy array has .ctypes property, returning aptr that can be easily passed as cffi function’s int64_t * argument. But single-value variables b and c don’t have same .ctypes property to get pointer of type int64_t * (compile error Unknown attribute 'ctypes' of type class(int64)). Neither I found any special function to get such pointer.

Maybe such address-getting special function exists somewhere in Numba’s library, can anyone point me please to such function? Otherwise it would be nice to implement such.

As a workaround I was creating single-element Numpy array, copying int value into this element and then passing arr[0:1].ctypes to function. After function completes I copy value from array back to single-value variable. But this workaround is very inefficient especially if to create array every time.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:1
  • Comments:13 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
gmarkallcommented, Sep 28, 2021

I think it will be difficult in general in Numba to support taking the address of variables, because we don’t have a well-defined concept of storage classes for variables in the frontend of Numba, unlike in C - so we’d need to do a lot of work to catch edge cases where a variable might not necessarily have any storage allocated.

@gmarkall Can you give an example, where you expect issues, with the code from above? Isn’t cgutils.alloca_once_value() a lightweight workaround (allocating stack memory, copying the provided value to this allocated stack-memory and returning a pointer to that stack memory), or do I get something wrong here?

I don’t expect issues with this. My concern was around more general support for taking the address of variables, where you do something like:

x = 2
addr = get_address_of(x)  # addr = &x
c_function(addr)          # Does *addr = 3
print(x)                  # prints 3
2reactions
max9111commented, Sep 24, 2021

@polkovnikov

I don’t see a reason why the following code should not work for (i8/i16/i32/i64/f32/f64) , but I haven’t tested it thoroughly.

from numba import types
from numba.extending import intrinsic
from numba.core import cgutils

@intrinsic
def val_to_ptr(typingctx, data):
    def impl(context, builder, signature, args):
        ptr = cgutils.alloca_once_value(builder,args[0])
        return ptr
    sig = types.CPointer(nb.typeof(data).instance_type)(nb.typeof(data).instance_type)
    return sig, impl

@intrinsic
def ptr_to_val(typingctx, data):
    def impl(context, builder, signature, args):
        val = builder.load(args[0])
        return val
    sig = data.dtype(types.CPointer(data.dtype))
    return sig, impl

Example

@nb.cfunc(types.void(nb.types.CPointer(nb.types.double)))
def add(x):
    wrapped_arr=nb.carray(x,shape=(1))
    wrapped_arr[0]+=1

@nb.njit()
def Test(A):
    x=val_to_ptr(A)
    add(x)
    y=ptr_to_val(x)
    return y

Test(15.)
#16.0
Read more comments on GitHub >

github_iconTop Results From Across the Web

Addresses and Pointers
Since the actual address of num is determined by your computer when you run the program, you may (or may not) get different...
Read more >
A C language pointer is a variable that contain the address of ...
The address of a variable is a non-negative integer number that uniquely identifies a specific location in the storage available to the program....
Read more >
Pointer Basics and Pass-By-Address
A pointer is a variable that stores a memory address. Pointers are used to store the addresses of other variables or memory items....
Read more >
CS31: Intro to C Structs and Pointers
It is just that the value of an array argument (the base address of the array) is different than the value of an...
Read more >
4. Basic Declarations and Expressions - Practical C++ ...
In computer programming you also need two things: data (variables) and instructions ... Let's see how these can be organized into a simple...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found