STATUS_STACK_BUFFER_OVERRUN crash on Application::Start with inappropriate SUBSYSTEM

Describe the bug STATUS_STACK_BUFFER_OVERRUN crash on Application::Start with inappropriate linker option SUBSYSTEM.

Steps to reproduce the bug

Steps to reproduce the behavior:

1. Create a Windows Console Application (C++/WinRT)
2. Write the following code in main.cpp

#include "pch.h"
#include <windows.h>
#include <winrt/Windows.UI.Xaml.h>

using namespace winrt;
using namespace Windows::Foundation;
using namespace Windows::UI::Xaml;

int main() {
	winrt::init_apartment();
	Application::Start([](auto&&) {});
	return 0;
}

1. Build and run the program
2. The exit code is -1073740791, which is 0xc0000409, STATUS_STACK_BUFFER_OVERRUN
3. Go to project property, linker, system, set subsystem to /SUBSYSTEM:WINDOWS
4. Write the following code in main.cpp

#include "pch.h"
#include <windows.h>
#include <winrt/Windows.UI.Xaml.h>

using namespace winrt;
using namespace Windows::Foundation;
using namespace Windows::UI::Xaml;

int __stdcall wWinMain(HINSTANCE, HINSTANCE, PWSTR, int)
{
	winrt::init_apartment();
	Application::Start([](auto&&) {});
	return 0;
}

1. Build and run the program
2. The exit code is no more -1073740791

Expected behavior Throw a user-friendly error like APPMODEL_ERROR_NO_PACKAGE.

Screenshots

Version Info OS version: 1903 (build 18362.836)

NuGet package version: Microsoft.Windows.CppWinRT 2.0.200602.3 (Also reproducible with winrt-rs 0.7)

Windows 10 version	Saw the problem?
Insider Build (xxxxx)
November 2019 Update (18363)
May 2019 Update (18362)	Yes
October 2018 Update (17763)
April 2018 Update (17134)
Fall Creators Update (16299)
Creators Update (15063)

Device form factor	Saw the problem?
Desktop	Yes
Mobile
Xbox
Surface Hub
IoT

Additional context I understand this is a rare scenario that nobody would do, but STATUS_STACK_BUFFER_OVERRUN does not make sense anyway.

[Austin-Lamb]

I hadn’t thought of that scenario, it’s an interesting one. I usually think of GUI and console apps as two separate universes.

Thanks for bringing that one up, I’ll reactivate this issue and leave it on the backlog, but it’s low-priority at this time.

[kennykerr]

This was first reported here: https://github.com/microsoft/winrt-rs/issues/214

The issue is that Xaml must report the invalid use of its API as a WinRT error so that languages with strong safety guarantees (like Rust) can make use of Xaml APIs.

Here is a minimal repro. The following C++/WinRT code should not crash but instead report some error message:

#include "winrt/Windows.UI.Xaml.h"

int main()
{
    try
    {
        winrt::Windows::UI::Xaml::Application::Start([](auto&&) {});
    }
    catch (winrt::hresult_error const& e)
    {
        printf("error: %ls\n", e.message().c_str());
    }
}

Instead, this code causes a failfast:

Windows.UI.Xaml.dll!00007FFF52667FC0: (caller: 00007FF759BC2F69) FailFast(1) tid(46fc) 8000FFFF Catastrophic failure
Unhandled exception at 0x00007FFF52667FC0 (Windows.UI.Xaml.dll) in scratch.exe: Fatal program exit requested.