Unity's IL2CPP // AOT compilation mode compatibility

Hi! ✋

Context

We’ve been using EF Core in our Unity3D app for more than a year. In the Unity Editor (which runs .NET 4.6) EF Core 1.1.3 works great, as long as we manually add the required .NET dependencies. The UWP export (HoloLens in our case) doesn’t work OOTB because the Unity’s ReferenceRewriter executable fails find some dependencies. We worked around the problem by adding the EF Core DLLs later in the build pipeline, in the UWP project generated by Unity; this way the RefRewriter doesn’t see any EFCore bits and doesn’t fail the build. A bit of a ‘hack’, but hey it works. That’s how we managed to use EFCore on Unity targeting UWP.

Issue

Now Unity is forcing developers to migrate to their IL2CPP scripting backend as they are about to drop .NET Scripting Backend support. Also, Unity supports .NET Standard 2 and the latest UWP SDK only in IL2CPP mode, so we basically have to migrate to IL2CPP. However, it seems that EF Core 2.0 and IL2CPP/AOT do not go well together.

Here we provide two ways to reproduce the problem. The first one might give more information about the issue, but the second one is much faster to reproduce, though the logs won’t tell you anything meaningful.

Steps to reproduce

1. By targeting iOS

Tested from May to June 2018

Details

• Unity v2018.2.*
• iOS player enabled
• IL2CPP scripting backend
• .NET Standard 2.0 compatibility
• Entity Framework Core 2.* with SQLite provider (also tested with InMemory for testing, IIRC)

iOS forbids code generation and IIRC that was the primary reason Unity worked on their IL2CPP system. But we rapidly noticed that this AOT compiler can’t manage to build EFCore properly as it seems to heavily rely on runtime code generation and reflection.

When trying to make EFCore work on iOS (which means by using IL2CPP), we ran into all sorts of exceptions. Here are a few of them:

NotSupportedException: /Users/builduser/buildslave/unity/build/External/il2cpp/il2cpp/libil2cpp/icalls/mscorlib/System.Reflection.Emit/AssemblyBuilder.cpp(20) : Unsupported internal call for IL2CPP:AssemblyBuilder::basic_init - System.Reflection.Emit is not supported.

---

Rethrow as TypeInitializationException: The type initializer for 'Microsoft.EntityFrameworkCore.Metadata.Internal.ClrAccessorFactory<Microsoft.EntityFrameworkCore.Metadata.Internal.IClrPropertySetter>' threw an exception.

---

ExecutionEngineException: Attempting to call method ‘Microsoft.EntityFrameworkCore.Metadata.Internal.ClrPropertySetterFactory::CreateGeneric’ for which no ahead of time (AOT) code was generated.

If necessary, I might be able to provide more detailed callstacks. We tried many different ways to make IL2CPP compile EFCore properly but as you can see, we couldn’t. One way or another, something fails, whether it’s the lack of generated code ahead of time, or a forbidden API (System.Reflections.Emit), or something else.

2. In the Editor

Tested and verified a week ago

Details

• Unity v2018.2.4
• WSA (UWP) player enabled
• IL2CPP scripting backend
• .NET Standard 2.0 compatibility
• UWP SDK 17134 (or close)
• Entity Framework Core 2.* with and without SQLite provider

Steps

• Create a .NET Standard 2 DLL “Test” and add the EntityFramework Core nuget package. We made sure there’s some EFCore actually shipped with the DLL by adding a dummy class inheriting from DbContext and adding a few test DbSets …
• Build project and copy the DLL in Unity’s Asset/Plugins folder
• Go back to Unity, which will take some time to process the DLL, as it always does when a DLL is updated
• This is what the Unity console says

Unloading broken assembly Assets/x64/Test.dll, this assembly can cause crashes in the runtime

This behaviour happens with some libraries such as Autofac and, as said, EFCore. The DLL works fine if EFCore nuget package is not installed in the “Test” DLL.

Question(s)

• Is it known that Entity Framework Core doesn’t work in AOT compilation mode, like NewtonSoft.Json and other libraries? Or it is supposed to work (I think I saw devs running EFCore on Xamarin on iOS, which would mean AOT, right? 🤔) and we’re doing it wrong?
• If it doesn’t currently work, is there any plans to provide an AOT (or IL2CPP) compatible version of Entity Framework Core? Many use Unity3D and many use EFCore, and I’m actually surprised I haven’t seen many talk about this issue until now, which makes me think (and hope) we’re doing some things wrong.

In the meantime, I’m also preparing to open a ticket on Unity’s support. I honestly still don’t know if the problems stands on EFCore’s part, Unity’s, or ours.

Thank you for helping!

[joshpeterson]

Hey, I’m a developer at Unity working on IL2CPP. Hopefully I can shed a bit of light on this issue.

Starting in Unity 2018.2, IL2CPP will use the Interpreter for Linq expressions, so some things have a chance of working. As discussed here, you will need to use a link.xml file, since the interpreter implementation uses reflection in internally, and the managed byte code stripper is a bit too aggressive with it.

This link.xml should be enough:

<linker>
  <assembly fullname="System.Core">
    <type fullname="System.Linq.Expressions.Interpreter.LightLambda" preserve="all" />
  </assembly>
</linker>

In Unity 2018.2, you’ll need to use the .NET Standard 2.0 Api Compatibility Level option in Unity. In Unity 2018.3 we’re also supporting this with the .NET 4.x Api Compatibility Level. In the future Unity 2019.1 version, you won’t need the link.xml file work around, the byte code stripper will be smart enough to keep this code around when it it used.

With that said, you might still hit some limitations of the IL2CPP AOT engine. Specifically, these are often related to the use of value types. So a snippet like this, which uses only reference types (a string), will work:

var body = Expression.Constant("Hey!");
var lambda = Expression.Lambda(body);
var function = lambda.Compile();
var result = function.DynamicInvoke();

But code which uses value types (like int or double) might run into AOT limitations. This occurs because IL2CPP shares implementations of generic type with reference type arguments, and so it can do a good bit of “runtime code generation” in these cases, since the only difference is the metadata, not the code.

IL2CPP does not have the capability to share generics with value type arguments yet, where Xamarin for iOS does. This is an area of future development for IL2CPP, but it is not ready for production yet.

Regarding the specific issues raised in the bug 1077758, I’m unsure about the cause of those errors. We’ll need to investigate that bug report further.

[bricelam]

Some of this work may overlap with #10963.

Our current plan is to focus on the .NET 5 linker/aot technology and hope it gets adopted by all .NET platforms like ASP.NET, Xamarin, UWP, and maybe even Unity someday. This was my favorite slide from dotNETConf last November:

In the meantime, we’d happily accept any well-written PRs that improve the IL2CPP (and overall Unity) experience with EF Core.