Type inference recursion error for self referencing types

I don’t know if this should be labeled as a bug or missing feature, but is worth exposing it. Basicaly, here I am trying to self reference(forward declaration) a structref without using the undocumented DeferredType.

example code

import numba as nb
from numba import jit, njit, types, typed, float64, int64, typeof
from numba.experimental import structref
from numba.core.extending import overload_method
from numba.experimental.structref import new, register

@structref.register
class PolygonStructType(types.StructRef):

    def preprocess_fields(self, fields):
        self.name = f"numba.PolygonStructType#{id(self)}" # temp name to allow Optional instantiation, this workaround can be replaced by extending StructRef or making StructRef to assign itself a name before preprocess_fields is called
        fields = tuple([
            ('value', types.Optional(int64)),
            ('parent', types.Optional(self)),
            ])

        return fields

polygon_struct_type = PolygonStructType(fields=(('value', types.Any), ('parent', types.Any)))

class PolygonStruct(structref.StructRefProxy):
    def __new__(cls, value, parent):
        return structref.StructRefProxy.__new__(cls, value, parent)

    @property
    def value(self):
        return PolygonStruct_get_value(self)

    @property
    def parent(self):
        return PolygonStruct_get_parent(self)

@njit
def PolygonStruct_get_value(self):
    return self.value

@njit
def PolygonStruct_get_parent(self):
    return self.parent

structref.define_proxy(PolygonStruct, PolygonStructType, ["value", "parent"])

@overload_method(PolygonStructType, "flip")
def PolygonStruct_flip(self):
    def impl(self):
        if self.value is not None:
            self.value = -self.value
    return impl

@njit
def test():
    # p = PolygonStruct(0, None)
    p = new(polygon_struct_type)
    p2 = new(polygon_struct_type)
    p2.value = 1
    p.parent = p2
    p.value = 3
    
    p.flip()
    return p.value

y = test()
print(y)

IMO, the usage of recursive algorithms in general is not recommended for production code. And forward declaration for types is an essential feature in strong typed languages. There is no real workaround for it. I don’t understand why numba does not have it already…

[rpopovici]

the error you are seing comes from traverse_types() in models.py. that’s where the inf recursion takes place

[sklam]

Regarding DeferredType, it was added for jitclass as part of an experiment just to prove that circular references work. However, it was not documented because there are still issues. The most important one is that Numba does not know how to collect circular references.

Anyway, I’m experimenting with a patch (https://github.com/numba/numba/pull/6209) so the following use of DeferredType will work:

@structref.register
class PolygonStructType(types.StructRef):
    def preprocess_fields(self, fields):
        # temp name to allow Optional instantiation, this workaround can be
        # replaced by extending StructRef or making StructRef to assign itself
        # a name before preprocess_fields is called
        self.name = f"numba.PolygonStructType#{id(self)}"

        # ======= new code =======
        # Create deferred type
        PolygonStructType_self = types.DeferredType()
        PolygonStructType_self.define(self)
        # ========= end =========
        fields = tuple(
            [
                ("value", types.Optional(int64)),
                ("parent", types.Optional(PolygonStructType_self)),
            ]
        )
        return fields