llvmlite assertion when using deferred type as typed dict values

From: https://numba.discourse.group/t/cannot-gettypeinfo-on-a-type-that-is-unsized-error-on-jitclass-with-typed-dict-variable-that-has-deferred-type-as-value-type/1191

import numba as nb
from collections import OrderedDict
from numba import deferred_type, optional
from numba.experimental import jitclass
from numba import types

node_type = deferred_type()

@jitclass([
  ('parent', optional(node_type)),
  ('children', types.DictType(types.int32, node_type)),
])
class TreeNode(object):
    def __init__(self, parent=None):
        self.parent = parent
        self.children = nb.typed.Dict.empty(types.int32, node_type)

node_type.define(TreeNode.class_type.instance_type)

t = TreeNode()
t2 = TreeNode(t2)

produces:

$ python repro.py 
reducer_override for <class 'type'>
python: /opt/conda/conda-bld/llvmdev_1637166008009/work/lib/IR/DataLayout.cpp:713: llvm::Align llvm::DataLayout::getAlignment(llvm::Type*, bool) const: Assertion `Ty->isSized() && "Cannot getTypeInfo() on a type that is unsized!"' failed.
Aborted (core dumped)

This may be an llvmlite bug or a combination of Numba doing something bad and llvmlite not catching / handling it very well, but I’m just opening the issue with the initial reproducer to start tracking it.

Note: Edits made to the the reproducer so it can run to completion once the Numba issue is resolved (the initial revision tried to pass 1 as parent, when it should have been a tree node type).

To do:

• Create an llvmlite reproducer / issue and investigate whether the assertion failure can easily be prevented.

[gmarkall]

The problem is that the context doesn’t ensure that the model for a deferred type has been defined prior to trying to get its actual size. The following hack:

diff --git a/numba/core/base.py b/numba/core/base.py
index 9622a3f09..54740d162 100644
--- a/numba/core/base.py
+++ b/numba/core/base.py
@@ -477,7 +477,10 @@ class BaseContext(object):
         The return value is a llvmlite.ir.Type object, or None if the type
         is an opaque pointer (???).
         """
-        return self.data_model_manager[ty].get_data_type()
+        dmm = self.data_model_manager[ty]
+        if isinstance(dmm, datamodel.models.DeferredStructModel):
+            dmm._define()
+        return dmm.get_data_type()
 
     def get_value_type(self, ty):
         return self.data_model_manager[ty].get_value_type()

allows the above code to run to completion, but may not be the right place to insert the fix.

[guilhermeleobas]

Current failure happens on llvmlite/bindings/targets.py::TargetData.get_pointee_abi_size. Below is the last five frames of the stacktrace:

[48]   /Users/guilhermeleobas/git/numba/numba/core/base.py(1220)wrapper()
-> return fn(*args, **kwargs)
[49]   /Users/guilhermeleobas/git/numba/numba/experimental/structref.py(305)codegen()
-> alloc_size = context.get_abi_sizeof(alloc_type)
[50]   /Users/guilhermeleobas/git/numba/numba/core/base.py(1105)get_abi_sizeof()
-> return ty.get_abi_size(self.target_data)
[51]   /Users/guilhermeleobas/miniconda3/envs/numba/lib/python3.9/site-packages/llvmlite/ir/types.py(53)get_abi_size()
-> return target_data.get_pointee_abi_size(llty)
[52] > /Users/guilhermeleobas/miniconda3/envs/numba/lib/python3.9/site-packages/llvmlite/binding/targets.py(156)get_pointee_abi_size()
-> size = ffi.lib.LLVMPY_ABISizeOfElementType(self, ty)

Assertion is triggered when Numba tries to get the size of the struct defined above:

 301         def codegen(context, builder, signature, args):
 302             # FIXME: mostly the same as jitclass ctor_impl()
 303             model = context.data_model_manager[inst_type.get_data_type()]
 304             alloc_type = model.get_value_type()
 305  ->         alloc_size = context.get_abi_sizeof(alloc_type)
 306
 307             meminfo = context.nrt.meminfo_alloc_dtor(
 308                 builder,
 309                 context.get_constant(types.uintp, alloc_size),
 310                 imp_dtor(context, builder.module, inst_type),
 311             )
 312             data_pointer = context.nrt.meminfo_data(builder, meminfo)
 313             data_pointer = builder.bitcast(data_pointer, alloc_type.as_pointer())
 314
 315             # Nullify all data
 316             builder.store(cgutils.get_null_value(alloc_type), data_pointer)
 317
 318             inst_struct = context.make_helper(builder, inst_type)
 319             inst_struct.meminfo = meminfo
 320
 321             return inst_struct._getvalue()
(Pdb++) alloc_type
<<class 'llvmlite.ir.types.LiteralStructType'> {i64, {%"deferred.4702211712.value", i1}, {%"deferred.4702211712.value", i1}}>