`getitem` access with `str` value to a struct array causes impl "no match"

Hopefully this isn’t a case of doing something obviously naive but, more or less the issue is that this compile error is the result of passing in a str argument (to an @njit-ed @staticmethod) to read a (numpy) struct-array via field name:

numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function getitem>) found for signature:

 >>> getitem(Record(time[type=float64;offset=0],open[type=float64;offset=8],high[type=float64;offset=16],low[type=float64;offset=24],close[type=float64;offset=32],volume[type=float64;offset=40],bar_wap[type=float64;offset=48];56;False), unicode_type)

There are 22 candidate implementations:
  - Of which 22 did not match due to:
  Overload of function 'getitem': File: <numerous>: Line N/A.
    With argument(s): '(Record(time[type=float64;offset=0],open[type=float64;offset=8],high[type=float64;offset=16],low[type=float64;offset=24],close[type=float64;offset=32],volume[type=float64;offset=40],bar_wap[type=float64;offset=48];56;False), unicode_type)':
   No match.

During: typing of intrinsic-call at /home/goodboy/repos/piker/piker/data/_pathops.py (562)

File "piker/data/_pathops.py", line 562:
    def path_arrays_from_ohlc(
        <source elided>
            close = q['close']
            index = float64(q[index_field])
            ^

---

Yes, i’m on 0.56.4.

---

source code function which triggers this is here: https://github.com/pikers/piker/blob/8ee964451b085dfb957160465018f4d4bd8ab696/piker/data/_pathops.py#L526

More or less it seems that there is an issue with the type signature matching of the input struct-array both in terms of inferred (unspecified in numba terms) type sig as well as a manual spec.

For eg. I’ve tried something like

    @staticmethod
    @njit(
        # TODO: need to construct this manually for readonly
        # arrays, see https://github.com/numba/numba/issues/4511
        (
            types.Array(
                numba_ohlc_dtype,
                1,
                'C',
                readonly=True,
            ),
            int64,
            types.unicode_type,
            optional(float64),
        ),
        nogil=True
    )
    def path_arrays_from_ohlc(
        data: np.ndarray,
        start: int64,
        index_field: str,
        bar_gap: float64 = 0.43,

    ) -> tuple[
        np.ndarray,
        np.ndarray,
        np.ndarray,
    ]:

And still get the same error.

---

Likely related issues include:

• https://github.com/numba/numba/issues/5756
• https://github.com/numba/numba/issues/5753
• https://github.com/numba/numba/issues/4098
• https://github.com/numba/numba/issues/4511 (data is this.)

[stuartarchibald]

Thanks for the report. I think this potentially demonstrates what it is you are seeing:

from numba import njit
import numpy as np

dt = np.dtype([('field_a', np.float32), ('field_b', np.int8)])

@njit
def foo(record, field):
    x = record[field] # What type is x ?
    return x

arr = np.ones(1, dt)

print(foo(arr, 'field_a'))

the issue is essentially that Numba sees the type of field as a types.unicode_type, it doesn’t know at compile time to which field in the dt dtype this corresponds. As a result it cannot work out a return type of the getitem call which manifests as a typing error similar to that in the OP.

The following case does work because the types can be resolved at compile time through the use of a compile-time-constant string that addresses the field:

from numba import njit
import numpy as np

dt = np.dtype([('field_a', np.float32), ('field_b', np.int8)])

@njit
def foo(record):
    # access 'field_a' with a compile time constant string...
    # the type of x is known to be the type of that specific field
    x = record['field_a']
    return x

arr = np.ones(1, dt)

print(foo(arr))

[stuartarchibald]

@goodboy I don’t think list based access to the fields is supported. If the list is declared as in the above, then it has an initial_value (the <iv=[...]> part of the type) which would potentially make it possible to implement this kind of indexing. If this is important to you then perhaps open a feature request? Many thanks.