Unrecoverable exceptions being frequently thrown

Description

Unrecoverable exceptions are being thrown by Puppeteer Sharp. This prevents any kind of retry or graceful exception handling and results in the process being terminated.

Complete minimal example reproducing the issue

    // Arrange
            var launchOptions = new LaunchOptions()
            {
                Headless = true,
            };

            var viewPortOptions = new ViewPortOptions()
            {
                Width = 1920,
                Height = 1080
            };

            var sites = new List<string>()
            {
                "wix.com",
                "facebook.com",
                "twitter.com",
                "google.com",
                "youtube.com",
                "instagram.com",
                "linkedin.com",
                "wordpress.org",
                "pinterest.com",
                "wikipedia.org",
                "wordpress.com",
                "blogspot.com",
                "apple.com",
                "adobe.com",
                "tumblr.com",
            }
    // Act
            try
            {
                await new BrowserFetcher().DownloadAsync(BrowserFetcher.DefaultRevision);
                using (var browser = await Puppeteer.LaunchAsync(launchOptions))
                {
                    //var context = await browser.CreateIncognitoBrowserContextAsync();
                    //var page = await context.NewPageAsync();
                    var page = await browser.NewPageAsync();
                    await page.SetViewportAsync(viewPortOptions);

                    foreach (var site in sites)
                    {
                        try
                        {
                            await page.GoToAsync($"http://{site}");
                            Console.WriteLine(await page.GetTitleAsync());
                            await page.ScreenshotAsync($"D:\\bin\\screenshots\\{site}.png");
                        }
                        catch (Exception exception)
                        {
                            // Catches most exceptions such as timeouts but does not catch others, see below.
                            Console.WriteLine($"Unable to take screenshot of: {site}. Exception: {exception.Message}");
                        }
                    }
                }
            }
            catch (Exception exception)
            {
                // Never enters the catch.
                Console.WriteLine($"Unable to proceed: {exception.Message}");
            }

Expected behavior:

The library to take a screenshot of the URL given.

Actual behavior:

Often crashes with exceptions which cannot be caught. There appears to be no common cause of the issue and it happens at random. The issue does not seem to be related to a particular site.

Same happens with normal and Incognito sessions.

Versions

• Which version of PuppeteerSharp are you using? 1.8.0
• Which .NET runtime and version are you targeting? .NET Core 2.1

Additional Information

The exceptions are lacking in details and so far the exceptions encountered are:

PuppeteerSharp.TargetCrashedException
  HResult=0x80131500
  Source=PuppeteerSharp
  StackTrace:
   at PuppeteerSharp.Page.OnTargetCrashed()
   at PuppeteerSharp.Page.<Client_MessageReceived>d__199.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

System.Collections.Generic.KeyNotFoundException
  HResult=0x80131577
  Message=The given key 'B99316384B7CE098D1E12423B7A2772B' was not present in the dictionary.
  Source=System.Private.CoreLib
  StackTrace:
   at System.ThrowHelper.ThrowKeyNotFoundException[T](T key)
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at PuppeteerSharp.NetworkManager.OnRequest(RequestWillBeSentPayload e, String interceptionId)
   at PuppeteerSharp.NetworkManager.OnRequestWillBeSent(RequestWillBeSentPayload e)
   at PuppeteerSharp.NetworkManager.<Client_MessageReceived>d__33.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

Here is the redirected output from Chrome. Again because this happens randomly I haven’t been able to determine an exact reproduction but it seems to happen more often when it navigates to a site that does not exist.

Unable to take screenshot of: wixsite.com. Exception: net::ERR_NAME_NOT_RESOLVED
 at http://wixsite.com at http://wixsite.com
Received fatal exception EXCEPTION_ACCESS_VIOLATION
Backtrace:
        GetHandleVerifier [0x00007FF9B8058D94+15468820]
        ovly_debug_event [0x00007FF9B6A08B5D+16084829]
        ovly_debug_event [0x00007FF9B6A088B7+16084151]
        ovly_debug_event [0x00007FF9B69FC185+16033157]
        ovly_debug_event [0x00007FF9B69FBA36+16031286]
        ovly_debug_event [0x00007FF9B69F764B+16013899]
        GetHandleVerifier [0x00007FF9B80583D0+15466320]
        GetHandleVerifier [0x00007FF9B77C89CC+6489420]
        GetHandleVerifier [0x00007FF9B77C8560+6488288]
        GetHandleVerifier [0x00007FF9B77C9A01+6493569]
        ovly_debug_event [0x00007FF9B626E19A+8111514]
        GetHandleVerifier [0x00007FF9B72C4E0F+1231247]
        GetHandleVerifier [0x00007FF9B72C3384+1224452]
        IsSandboxedProcess [0x00007FF9B71781BC+1557308]
        IsSandboxedProcess [0x00007FF9B718C3E6+1639782]
        IsSandboxedProcess [0x00007FF9B71781BC+1557308]
        ovly_debug_event [0x00007FF9B6CC0C4F+18935887]
        ovly_debug_event [0x00007FF9B6CC1045+18936901]
        IsSandboxedProcess [0x00007FF9B717AAF9+1567865]
        ovly_debug_event [0x00007FF9B6CD96D1+19036881]
        IsSandboxedProcess [0x00007FF9B713319E+1274654]
        ovly_debug_event [0x00007FF9B6C9A9D3+18779603]
        ovly_debug_event [0x00007FF9B6C9E9C6+18795974]
        ovly_debug_event [0x00007FF9B6C9A191+18777489]
        IsSandboxedProcess [0x00007FF9B716F27D+1520637]
        IsSandboxedProcess [0x00007FF9B716F194+1520404]
        ovly_debug_event [0x00007FF9B6C9CCAE+18788526]
        ChromeMain [0x00007FF9B5AB1111+273]
        Ordinal0 [0x00007FF7F23531CC+12748]
        Ordinal0 [0x00007FF7F235168D+5773]
        GetHandleVerifier [0x00007FF7F24557E2+793282]
        BaseThreadInitThunk [0x00007FFA016A13D2+34]
        RtlUserThreadStart [0x00007FFA036354F4+52]

As a side effect dozens of Chromium instances are left running, though that’s not really within scope of this issue.

[lloydjatkinson]

I’ve run a few hours of testing and the issue seems to have been resolved and no process termination 👍 Thanks!

I have however come across a new edge case. I’m not sure if it belongs in this issue or a new one though.

When in headless mode navigating to http://one.com causes the Windows/Console beep to occur - every time the site is hit (I’ve not encountered this on any other site). When not running in headless mode and hitting the same site and with SlowMo enabled, however, the console beeps are not emitted and I get the following exceptions (not in any particular order).

Let me know if this is better suited in a new issue!

System.InvalidOperationException
  HResult=0x80131509
  Message=Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.
  Source=System.Private.CoreLib
  StackTrace:
   at System.ThrowHelper.ThrowInvalidOperationException_ConcurrentOperationsNotSupported()
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.set_Item(TKey key, TValue value)
   at PuppeteerSharp.Browser.<CreateTargetAsync>d__63.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\Browser.cs:line 399
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PuppeteerSharp.Browser.<Connect_MessageReceived>d__60.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\Browser.cs:line 335

System.InvalidOperationException
  HResult=0x80131509
  Message=An attempt was made to transition a task to a final state when it had already completed.
  Source=System.Private.CoreLib
  StackTrace:
   at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
   at PuppeteerSharp.Frame.SetDefaultContext(ExecutionContext context) in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\Frame.cs:line 644
   at PuppeteerSharp.FrameManager.<OnExecutionContextCreatedAsync>d__44.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 247
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PuppeteerSharp.FrameManager.<_client_MessageReceived>d__39.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 178

System.IndexOutOfRangeException
  HResult=0x80131508
  Message=Index was outside the bounds of the array.
  Source=System.Collections
  StackTrace:
   at System.Collections.Generic.HashSet`1.Enumerator.MoveNext()
   at PuppeteerSharp.FrameManager.AddFrame(String framId, Frame frame) in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 305
   at PuppeteerSharp.FrameManager.OnFrameNavigated(FramePayload framePayload) in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 292
   at PuppeteerSharp.FrameManager.<_client_MessageReceived>d__39.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 162

System.IndexOutOfRangeException
  HResult=0x80131508
  Message=Index was outside the bounds of the array.
  Source=System.Collections
  StackTrace:
   at System.Collections.Generic.HashSet`1.AddIfNotPresent(T value)
   at System.Collections.Generic.HashSet`1.Add(T item)
   at PuppeteerSharp.Helpers.MultiMap`2.Add(TKey key, TValue value) in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\Helpers\MultiMap.cs:line 15
   at PuppeteerSharp.FrameManager.<GetFrameAsync>d__54.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 376
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at PuppeteerSharp.FrameManager.<OnExecutionContextCreatedAsync>d__44.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 236
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PuppeteerSharp.FrameManager.<_client_MessageReceived>d__39.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 178

System.IndexOutOfRangeException
  HResult=0x80131508
  Message=Index was outside the bounds of the array.
  Source=System.Collections
  StackTrace:
   at System.Collections.Generic.HashSet`1.AddIfNotPresent(T value)
   at System.Collections.Generic.HashSet`1.Add(T item)
   at PuppeteerSharp.Helpers.MultiMap`2.Add(TKey key, TValue value) in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\Helpers\MultiMap.cs:line 15
   at PuppeteerSharp.FrameManager.<GetFrameAsync>d__54.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 376
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at PuppeteerSharp.FrameManager.<OnExecutionContextCreatedAsync>d__44.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 236
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PuppeteerSharp.FrameManager.<_client_MessageReceived>d__39.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 178

System.NullReferenceException
  HResult=0x80004003
  Message=Object reference not set to an instance of an object.
  Source=System.Private.CoreLib
  StackTrace:
   at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
   at System.Collections.Generic.Dictionary`2.ContainsKey(TKey key)
   at PuppeteerSharp.Browser.<CreateTargetAsync>d__63.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\Browser.cs:line 394
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PuppeteerSharp.Browser.<Connect_MessageReceived>d__60.MoveNext() in C:\Users\Lloyd\Source\Repos\lloydjatkinson\puppeteer-sharp\lib\PuppeteerSharp\Browser.cs:line 335

[kblok]

Could you pull from support/onrequest-changes again @lloydjatkinson?