question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Invalid offset in buffer format string generation, when trying to use PyOpenCl with Numpy

See original GitHub issue

I am porting over one of my projects from cpu to gpu and in the process I simply copied some of the code from http://documen.tician.de/pyopencl/howto.html However I keep getting a numpy error. Code:

import pyopencl as cl
import pyopencl.tools
import pyopencl.array
import numpy as np

def gpuTest():
    context = cl.create_some_context(interactive=False)
    queue = cl.CommandQueue(context)
    my_struct = np.dtype([("field1", np.int32), ("field2", np.float32)])
    my_struct, my_struct_c_decl = cl.tools.match_dtype_to_c_struct(context.devices[0], "my_struct", my_struct)
    my_struct = cl.tools.get_or_register_dtype("my_struct", my_struct)

    print(my_struct_c_decl)

    ary_host = np.empty(20, my_struct)
    ary_host["field1"].fill(217)
    ary_host["field2"].fill(1000)
    ary_host[13]["field2"] = 12

    print(ary_host)

    ary = cl.array.to_device(queue, ary_host)

    prg = cl.Program(context, my_struct_c_decl + """
    ...     __kernel void set_to_1(__global my_struct *a)
    ...     {
    ...         a[get_global_id(0)].field1 = 1;
    ...     }   
    ...     """).build()

    evt = prg.set_to_1(queue, ary.shape, None, ary.data)

    print(ary)


gpuTest()
input()

Here is the console output:

typedef struct {
  int field1;
  float field2;
} my_struct;


[(217, 1000) (217, 1000) (217, 1000) (217, 1000) (217, 1000) (217, 1000)
 (217, 1000) (217, 1000) (217, 1000) (217, 1000) (217, 1000) (217, 1000)
 (217, 1000) (217, 12) (217, 1000) (217, 1000) (217, 1000) (217, 1000)
 (217, 1000) (217, 1000)]
Traceback (most recent call last):
  File "C:\Users\\Desktop\Programming\Python Programming\Pygpu\main.py", line 36, in <module>
    gpuTest()
  File "C:\Users\\Desktop\Programming\Python Programming\Pygpu\main.py", line 22, in gpuTest
    ary = cl.array.to_device(queue, ary_host)
  File "C:\Python34\lib\site-packages\pyopencl\array.py", line 1680, in to_device
    result.set(ary, async=async)
  File "C:\Python34\lib\site-packages\pyopencl\array.py", line 670, in set
    is_blocking=not async)
  File "C:\Python34\lib\site-packages\pyopencl\__init__.py", line 1051, in enqueue_copy
    return _cl._enqueue_write_buffer(queue, dest, src, **kwargs)
RuntimeError: This should never happen: Invalid offset in buffer format string generation. Please 
report a bug to the Numpy developers.
Press any key to continue . . .

I cant seem to find anyone else who has had this kind of problem, is there something I could have done wrong when installing? Thank you in advance

Issue Analytics

  • State:closed
  • Created 8 years ago
  • Comments:34 (18 by maintainers)

github_iconTop GitHub Comments

1reaction
crackwitzcommented, May 29, 2017

https://gist.github.com/crackwitz/e13b600a1a0158cc3301918b7edf5200

if I access the mapped array fields directly, no error…

edit: the numpy docs only mention “union” once with this meaning, and that’s where this arbitrary struct definition is explained.

This form also makes it possible to specify struct dtypes with overlapping fields, functioning like the ‘union’ type in C. This usage is discouraged, however, and the union mechanism is preferred.

however, nowhere is explained what that is and how to actually do it. I’m left with doing it this way.

1reaction
sebergcommented, Jun 27, 2015

No, this is right. I think the weird thing is that the fields are in the wrong order maybe? The offset 4 is before the offset 0, is this a legal way to index fields? I.e. why does the int field have an offset of 4? it should be 0, no? and the float field should have 4?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Using offsets into the buffer in PyOpenGL calls
If named element array buffer object is stated in the vertex array object, then the last parameter of glDrawElements is treated as a...
Read more >
numpy.frombuffer — NumPy v1.24 Manual
Start reading the buffer from this offset (in bytes); default: 0. ... Reference object to allow the creation of arrays which are not...
Read more >
PySchedCL 1.0 documentation
[docs]def ctype(dtype): """ Convert a string datatype to corresponding Numpy datatype. User can also define new datatypes using user_defined parameter.
Read more >
OpenCL Runtime: Memory - pyopencl 2022.2.4 documentation
This style permits using memory areas with pyopencl 's SVM interfaces even if they were allocated outside of pyopencl . Since passing a...
Read more >
pyopencl/tools.py at main · inducer ...
Memory allocation (e.g. in the form of the :func:`pyopencl.Buffer` constructor). can be expensive if used frequently. For example, code based on.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found