F# Support in .NET Native and UWP

Update

See this comment

There is now an official solution by Microsoft to put a .NET/Win32 app in the Windows Store that does not require the use of UWP APIs, Xamarin, React Native, or even changes to your existing F#/.NET code, because they removed the requirement of UWP APIs and .NET Native for Windows Store apps: Desktop to UWP Bridge.

Microsoft has basically modified the Windows Store in Windows 10 to support “Desktop” apps that are not using the UWP APIs. They just need to be packaged as UWP apps and they will even run on ARM devices (through x86 emulation) that run the recently announced Windows 10 for ARM in the future,

Original Issue

We’ve been working on adding F# support to the .NET Native compiler. When this work is finished, you’ll be able to write Portable Class Libraries in F# that can be used from Universal Windows apps, which are compiled with .NET Native. Since the .NET Native compiler is closed source, this issue is here as a way to share status regularly from the Microsoft team.

As some background, .NET Native is a compiler that turns .NET IL code directly into native code ahead-of-time, rather than just-in-time. If you hear “.NET Native doesn’t support F#,” what’s meant is “.NET Native doesn’t support all of the IL produced by the Visual F# compiler.”

Outline of the work required

1. Identify the specific work required for .NET Native to support F# - In progress
2. Implement the corresponding bug fixes and new features in the .NET Native and Visual F# compilers - In progress
3. Implement whatever is necessary in Visual Studio to support source code, projects, and assemblies for F# Portable Class Libraries in Universal Windows apps - Not yet started.

Identifying the specific work required

There are two kinds of issues we’re looking into: issues which we’ve confirmed by observing failing .NET Native compilations of F# IL; and issues we expect to have, but haven’t yet confirmed.

David (@numberbycolors) and Kevin (@KevinRansom) have spent several weeks working to compile the fsharp test suite with .NET Native. Most tests compiled without problems: 18 of the 23 unit tests with the coreclr tag. Several issues have been encountered in the other 5 failing tests:

• A bug where a 17 layer-deep nested generic type (eg. A<B<C<D<…<T>>>>>) would cause .NET Native to infer the type was infinitely deep and fail
• A bug where enum members marked ‘internal’ would cause .NET Native to fail.

There are other issues which we expect to encounter, but still need to confirm in other tests:

• The .tail IL instruction isn’t yet supported by .NET Native
• Tail call optimization, which is needed for the correctness of some F# programs, isn’t yet supported by .NET Native.

Implement the corresponding features and fixes

The next step is implementing the corresponding fixes and features in the .NET Native and Visual F# compilers. This work will be prioritized against the other work going on in .NET Native and Visual F#. Since the total amount of work is still unknown, we don’t have an estimate for when this F# support will be finished and delivered.

Some of these features and fixes have already been underway for months. While we’ve been investigating F#-specific issues in .NET Native, the team has continued to improve .NET Native. One feature known as “universal shared generics” is likely to have improved .NET Native’s support for F#, even without that being an explicit goal of the feature. Learn more about universal shared generics here.

Implement Visual Studio changes

Once the changes are made to the .NET Native and Visual F# compilers, the Visual F# team will need to do some work in Visual Studio to make sure F# projects behave correctly in Universal Windows app solutions.

[cartermp]

Update time!

Clearing up .NET Native

.NET Native is, for now, the proprietary native compilation technology used for AoT compilation of UWP apps. No other scenario is supported at this time.

The .NET Native toolchain team is tackling a huge number of core issues that affect everything (not just supporting one particular language), and these are a high priority for them to fix. These issues affect key internal and external customers.

Does the .NET Native Team Care about F#

In a word, YES. The issues which have been found are on their backlog and they have every intention of adding support for them. But to be clear, these are not trivial to support. Tailcall optimization, for example, is a big engineering effort that will take significant time to build and get right.

So where is F# on .NET Native Support Right now?

Still in Investigation Mode. We get recent internal builds of the toolchain that we run tests on, we add tests to try and see what else we can break, and we report issues. These issues get logged into the .NET Native engineering backlog as soon as they come up.

What can the F# community do to help?

The best thing you can do is to ask for F# support (such as UserVoice) and provide examples of how you would like to write F# for UWP applications.

If you have a need to write F# code for UWP apps, please reach out to me! You can email me directly (phcart at microsoft dot com) or DM me on Twitter. I want to get as much concrete info about needs as possible, as I can take these directly to the .NET Native team.

Does Microsoft Care about F#?

Yes! Yes yes yes yes yes! And more love for F# is coming, believe me - and not just in the OSS world. But things are taking time because so many things - the Visual Studio platform, the entirety of .NET - have been making major shifts in direction and huge engineering investments.

Okay, fine, so how about some dates?

As far as UWP/.NET Native support is concerned, I don’t have a date for when F# support is completed. It won’t be this year, though. As mentioned earlier - for example, getting tailcall optimization fully supported is going to be nontrivial and will take some time.

I’ll reiterate this - F# is on the .NET Native roadmap, smaller issues have already been worked on, the big issues are logged, and the .NET Native team intends to fully support F#. But .NET Native and UWP are far from done, and there are massive engineering investments yet to come.

Hopefully this clears a few things up. Thanks for being understanding. If F# support for building UWP apps is essential to you, please send me at email (or DM me on Twitter) so we can set up a chat.

[dsyme]

Part of the background is that, historically, some teams have implemented subsets of the .NET (by which I mean the ECMA 335 Standard). These subsets have historically had various limitations not subject to a public technical governance process. It’s a problem with .NET technical governance that I believe is now being more systematically dealt with.

Here are some personal recollections about this (my opinions, not Microsoft’s)

AFAICR this started back with the variant of .NET that was supported in-process in SQL Server. There were random restrictions imposed by some SQL Server “verifier” that checked that, for example, static class constructors didn’t access fields outside their class. That restriction wasn’t documented anywhere, and meant that the ECMA 335 standard wasn’t supported correctly, and so innocent tool developers (such as those writing code generators or compilers) simply ended up generating invalid IL. Some assemblies loaded into SQL Server. Some didn’t. You just had to suck it and see, and put random hacks into your compiler to make it generate code that satisfied that particular system.

This pattern has repeated itself over the years, e.g.

• Early versions of 64-bit support for .NET Framework implemented tailcalls in very inconsistent ways
• Early versions of Silverlight either didn’t do tailcalls properly or did them inconsistently
• Every version of .NET for Windows Phone and Windows App Store has had limitations.
• .NET Micro Framework had various limitations (no generics)
• All native code generators apart from NGEN.EXE had limitations, e.g. UWP
• NET Core 1.0 had various limitations
• The version of .NET used by Midori internally changed some things

Mono has had some issues but has generally been pretty good at implementing the whole standard (ok, there are still some issues with some tailcalls), and trying hard to do that for Xamarin targets too. That’s one reason why F# on Xamarin fares better than F# on these other targets. (kudos to @migueldeicaza and team)

We at Microsoft have sometimes not seen completeness (i.e. implementing the whole ECMA spec) and design integrity (i.e. no undocumented adhoc limitations) as sufficiently important. These things do matter: every time you miss something in the implementation of a standard, a hundred puppies die (joke - I’ve nothing against puppies 😃 )

Some problems also lie with ECMA 335 itself. Some features in that standard are hard or impossible to implement, especially if you aim to do native code generation without a JIT (though again Mono/Xamarin have achieved this pretty well). Tailcalls can be quite tricky, and some aspects of generics too. We should have predicted this. We should have avoided using these constructs in .NET libraries (e.g. generic virtual methods) or as an essential part of F# code generation (e.g. tailcalls) But equally, I do wish that we had adjusted the standard appropriately - and there should have been a public technical governance process in place to allow that to happen.

Of course, standardization and overly rigid technical governance can have problems too. But you can also get situations where individual teams are empowered to cut features to meet deadlines, or to add restrictions to simplify matrix testing, and that needs an over-arching technical governance process to ensure completeness and integrity.

Things are improving greatly with .NET Standard 2.0. But things like tailcalls or proper support for .NET Generics or whatever can still slip through the cracks. F# can get hit badly by this, though there are normally good workarounds - Xamarin, Fable, VSCode etc.

I hope that going forward the .NET architects now take a much more robust point of view about what it means to implement .NET Standard 2.0 properly and completely. This applies particularly to native code generation scenarios (including tailcalls and full support for generics).

And if, for some reason, it is impossible to implement .NET ECMA 335 properly, then the .NET architects should clarify some “officially recognized subset” of .NET and call it “.NET Standard Native” or whatever, and supply an automated verifier to check if code lies in that subset. But it should be an official subset, not adhoc.