question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Incompatibility in handling of SIGSEGV between ChakraCore and CoreCLR

See original GitHub issue

I discovered an incompatibility in handling of the segmentation fault signal between the ChakraCore engine and CoreCLR. The issue IMO impacts all recent ChakraCore versions. It is triggered by a segmentation fault in user code (e.g. any NullReferenceException) in a CoreCLR-driven process and manifests itself as a stack overflow / stack smashing that crashes the process.

The following occurs after the ChakraCore engine is loaded into a CoreCLR process when a segmentation fault is triggered:

Process is terminating due to StackOverflowException.
Aborted (core dumped)

The core dump contains the following stack:

frame #0: 0x00007ff9a96d7fff libc.so.6`gsignal + 207
frame #1: 0xfffffffe7fffffff
frame #2: 0x00007ff9a979e1f7 libc.so.6`__fortify_fail + 55
frame #3: 0x00007ff9a979e1c0 libc.so.6`__stack_chk_fail + 16
frame #4: 0x00007ff44c0e94a2 libChakraCore.so`sigsegv_handler(int, siginfo_t*, void*) + 171

When debugging the crash in lldb we can see a more informative stack:

* thread #1, name = 'dotnet', stop reason = signal SIGABRT
  * frame #0: libc.so.6`__GI_raise(sig=2) at raise.c:51
    frame #1: libc.so.6`__GI_abort at abort.c:79
    frame #2: 0x00007f7616872e73 libcoreclr.so`PROCAbort + 19
    frame #3: 0x00007f761683c262 libcoreclr.so`sigsegv_handler(int, siginfo_t*, void*) + 338
    frame #4: 0x00007f760df08187 libclrjit.so`sigsegv_handler(int, siginfo_t*, void*) + 247
    frame #5: 0x00007f757571146b libChakraCore.so`sigsegv_handler(int, siginfo_t*, void*) + 171

According to my investigation, the stack overflow is only virtual. CoreCLR registers its SIGSEGV handler with the the SA_ONSTACK flag that instruct the kernel to execute the handler on a separate stack. On the other hand, ChakraCore registers its handler without the flag. The discrepancy is IMO the reason why the CoreCLR SO detection malfunctions and aborts the process. It seems that the SO detector cannot deal with the fact that the SIGSEGV handler is not executing on the separate stack CoreCLR prepared during PAL initialization.

A small repro project can be downloaded here. Crashes consistently on Ubuntu 18.04.

Once I patched the registration of the SIGSEGV handler in ChakraCore (to align it with the CoreCLR version) the crash was gone:

73c73
< static void handle_signal(int signal_id, SIGFUNC sigfunc, struct sigaction *previousAction);
---
> static void handle_signal(int signal_id, SIGFUNC sigfunc, struct sigaction *previousAction, int additionalFlags = 0);
120c120
<     handle_signal(SIGSEGV, sigsegv_handler, &g_previous_sigsegv);
---
>     handle_signal(SIGSEGV, sigsegv_handler, &g_previous_sigsegv, SA_ONSTACK);
579c579
< void handle_signal(int signal_id, SIGFUNC sigfunc, struct sigaction *previousAction)
---
> void handle_signal(int signal_id, SIGFUNC sigfunc, struct sigaction *previousAction, int additionalFlags)
583c583
<     newAction.sa_flags = SA_RESTART;
---
>     newAction.sa_flags = SA_RESTART | additionalFlags;
591a592,598
>
> #ifdef INJECT_ACTIVATION_SIGNAL
>     if ((additionalFlags & SA_ONSTACK) != 0)
>     {
>         sigaddset(&newAction.sa_mask, INJECT_ACTIVATION_SIGNAL);
>     }
> #endif

I don’t know whether the issue lies within the CoreCLR SO detector or ChakraCore. Either the handler registration has to be reconciled or the SO detector has to account for the fact that an incompatible SIGSEGV handler may be registered later in the process.

The issue has been previously reported as #4893 and #5781. There may be also another incompatibility between ChakraCore and CoreCLR that prevents analysis of managed core dumps (maybe the ChakraCore PAL is an incompatible copy-paste from CoreCLR?).

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:6
  • Comments:21 (10 by maintainers)

github_iconTop GitHub Comments

3reactions
rhuanjlcommented, Apr 2, 2020

For another thought - in the master branch of CC we could try updating signal.cpp to the latest version from the CoreCLR and seeing if that solves this issue, our current version (from the CORECLR of about 4 years ago is vastly different to the latest) - in general it may be worth updating a lot of the PAL files to newer versions.

EDIT: in the alternative/or additionally we could explore deleting/removing a lot of this code; I’m unsure if ChakraCore internally needs this signal handling, not sure if someone else could weigh in on that? For reference:

  1. a lot of it was removed some time ago with a comment left to consider further clean up later: https://github.com/microsoft/ChakraCore/pull/2086
  2. much of it was added apparently to enable building of GCStress - a testing tool built alongside CC rather than CC itself - it’s unclear why code added for that is included in CC itself (EDIT2: looks like it is used more heavily than the PR comments indicate on linux, wholesale removal is unlikely to be possible): https://github.com/microsoft/ChakraCore/pull/408
2reactions
Taritsyncommented, Apr 2, 2020

@divmain @rhuanjl I know a lot of people who, due to this error, cannot use the ChakraCore library in .NET Core applications on Linux. Yesterday, Nikita Tsukanov (@kekekeks) suggested that I build a native assemblies with the Tomáš Deml’s patch applied, source code of which is contained in this discussion, and test it. For testing, I used my test suite, which I ran on 64-bit versions of Windows 10, Ubuntu 16.04.1 and macOS 10.12. All tests worked correctly, and most importantly, periodically occurring errors on Ubuntu completely disappeared. Therefore, I included these assemblies in the JavaScript Engine Switcher version 3.4.5. I suggest adding the Tomáš Deml’s patch along with security patches to the next release of ChakraCore v1.11.

@tomasdeml, I think it makes sense to submit your patch as PR.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Incompatibility in handling of SIGSEGV between ChakraCore ...
I discovered an incompatibility in handling of the segmentation fault signal between the ChakraCore engine and CoreCLR.
Read more >
Segmentation fault handling - Stack Overflow
The behavior of a process is undefined after it returns normally from a signal-catching function for a [XSI] SIGBUS, SIGFPE, SIGILL, or SIGSEGV...
Read more >
JavaScriptEngineSwitcher.ChakraCore 3.4.5 - NuGet Gallery
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of...
Read more >
mono-core-6.8.0-bp152.4.6 - SUSE Package Hub -
Mac.dll. * gh#mono/mono#7472 - Incorrect NullReferenceException when using extension methods on null instances as Action or Func arguments or variables. * Sat ...
Read more >
ibm-data-db2-6.8.0-bp153.1.194 RPM for aarch64 - RPMFind
from file list for aarch64 and ppc64/ppc64le * Tue Feb 04 2020 ... 0x01 not handled * gh#mono/mono#10309 - Mono issuing SIGSEGV running...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found