TestCafe intermittently throws "Cannot read property '0' of null"

What is your Scenario?

When we use TestCafe version 1.14.2, our test suites pass consistently. When we attempt to upgrade to a newer version, even TestCafe 1.18.6, our tests intermittently fail. We would like to be able to upgrade to the latest version of TestCafe.

What is the Current behavior?

When we try to update to versions of TestCafe greater than 1.14.2, many different scenarios in our test suite trigger the “Cannot read property ‘0’ of null” error.

When using page objects, the stack is usually unhelpful:

 ✖ User has ability to insert a Creative Assistant block into their email campaigns (screenshots:
 /Volumes/casesensitive/git/mc_testcafe/reports/2022-05-05_09-09-26/1/errors/1.png)

   1) TypeError: Cannot read property '0' of null

      Browser: Chrome 100.0.4896.127 / macOS 10.15.7
      Screenshot: /Volumes/casesensitive/git/mc_testcafe/reports/2022-05-05_09-09-26/1/errors/1.png

         270 |        this._handlePlainObject(val);
         271 |};
         272 |
         273 |DecodingTransformer.prototype.transform = function () {
         274 |    this.visitedRefs[0] = true;
       > 275 |    this._handleValue(this.references[0], this.references, 0);
         276 |
         277 |    return this.references[0];
         278 |};
         279 |
         280 |

         at DecodingTransformer.transform (/Volumes/casesensitive/git/mc_testcafe/node_modules/replicator/index.js:275:38)
         at module.exports.Replicator.decode (/Volumes/casesensitive/git/mc_testcafe/node_modules/replicator/index.js:577:24)



 1/1 failed (2m 47s)

It seems that the error becomes slightly more helpful when not creating any abstractions, but the error still unexpected:

 Running tests in:
 - Chrome 100.0.4896.127 / macOS 10.15.7

 Using the Creative Assistant in the Nuni email editor
 ✖ TestCafe Demo Test (screenshots: /Volumes/casesensitive/git/mc_testcafe/reports/2022-05-05_09-49-25/1/errors/1.png)

   1) TypeError: Cannot read property '0' of null

      Browser: Chrome 100.0.4896.127 / macOS 10.15.7
      Screenshot: /Volumes/casesensitive/git/mc_testcafe/reports/2022-05-05_09-49-25/1/errors/1.png

         27 |      .expect(loadingFrame.classNames)
         28 |      .notMatch(/active/, "container loading frame active", {timeout: 60000})
         29 |      .switchToIframe(Selector("#fallback:not([title *='Loading'])").with({visibilityCheck: true, timeout: 30000}));
         30 |  }
         31 |  await t
       > 32 |    .expect(Selector("[data-dojo-attach-point='itemContent'] img").with({ visibilityCheck: true }).exists).ok({timeout: 20000})
         33 |    .click(Selector("button").withText("Design Email"))
         34 |    .click(Selector("button").withAttribute("data-dojo-attach-point", "selectInlineEditorButton"))
         35 |});
         36 |

         at <anonymous> (/Volumes/casesensitive/git/mc_testcafe/browser_tests/campaigns/email/nuni/TestCafeDemoTests.js:32:13)
         at asyncGeneratorStep (/Volumes/casesensitive/git/mc_testcafe/browser_tests/campaigns/email/nuni/TestCafeDemoTests.js:1:230)
         at _next (/Volumes/casesensitive/git/mc_testcafe/browser_tests/campaigns/email/nuni/TestCafeDemoTests.js:1:568)



 1/1 failed (1m 34s)

What is the Expected behavior?

Tests should consistently pass with TestCafe 1.18.6 just as they do with TestCafe 1.14.2.

What is your public website URL? (or attach your complete example)

https://mailchimp.com/login

What is your TestCafe test code?

import {Selector} from "testcafe";

fixture`TestCafe Demo Test`
  .beforeEach(async t => {
    await t
    .maximizeWindow()
    .navigateTo("https://mailchimp.com/login")
      .expect(Selector("div").withAttribute("data-mc-el", "loginAnnouncementAnimation")
        .with({ visibilityCheck: true }).exists).ok()
      .typeText(Selector("#username"), "abc123") // Enter your username here
      .typeText(Selector("#password"), "abc123") // Enter your password here
      .click(Selector("button[value='log in']"));
  })

test('Create email campaign', async t => {
  await t.click(Selector("li a").withAttribute("href", "#/create-campaign"))
    .click(Selector("div[role = 'listitem']").withText("Email"))
    .click(Selector("a span").withExactText("Regular"))
    .typeText(Selector("input").withAttribute("name", "textValue"), "Test Campaign")
    .click(Selector("button").withText("Begin"))
    .switchToMainWindow();

  const loadingFrame = Selector("[class *= 'frame'] [class *= 'loading']");
  await loadingFrame();
  if (await loadingFrame.exists) {
    await t
      .expect(loadingFrame.classNames)
      .notMatch(/active/, "container loading frame active", {timeout: 60000})
      .switchToIframe(Selector("#fallback:not([title *='Loading'])").with({visibilityCheck: true, timeout: 30000}));
  }
  await t
    .expect(Selector("[data-dojo-attach-point='itemContent'] img").with({ visibilityCheck: true }).exists).ok({timeout: 20000})
    .click(Selector("button").withText("Design Email"))
    .click(Selector("button").withAttribute("data-dojo-attach-point", "selectInlineEditorButton"))
});

Your complete configuration file

{ “reporter”: [ { “name”: “spec” }, { “name”: “html”, “output”: “reports/myreports.html” }, { “name”: “json”, “output”: “reports/report.json” } ], “skipJsErrors”: true, “screenshots”: { “path”: “reports/”, “fullPage”: true, “pathPattern”: “${DATE}_${TIME}/${TEST_INDEX}/${FILE_INDEX}.png”, “takeOnFails”: true }, “src”: “browser_tests/”, “browsers”: [“chrome”], “concurrency”: 4, “allowMultipleWindows”:true, “videoPath”: “reports/videos”, “videoOptions” : { “singleFile”: true, “failedOnly”: true, “pathPattern”: “${TEST_INDEX}/${USERAGENT}/${FILE_INDEX}.mp4” }, “color”: true, “clientScripts”:[ {“module”:“axe-core/axe.min.js”}, {“content”: “window.addEventListener(‘error’, function (e) {console.error(e.message);});”} ] }

Your complete test report

I tried attaching the html report, but Github gave me an error when I tried to create the issue, stating that the body is too long. Please letm know what additional content from the html report would be helpful.

Screenshots

Steps to Reproduce

1. Log in (I’m unable to provide you with an account, but you all have created free accounts to help us previously)
2. Click “Create”
3. Select “Email”
4. Select “Regular”
5. Enter campaign title
6. Select “Begin”
7. Select “Design Email”
8. Select “New Editor”

TestCafe version

1.18.6

Node.js version

14.17.2

Command-line arguments

testcafe -c 1 -T “TestCafe Demo Test”

Browser name(s) and version(s)

Chrome 100.0.4896.127

Platform(s) and version(s)

macOS 10.15.7

Other

The tests do not fail 100% of the time, so you may need to rerun the test demo script included in this issue several times before the error appears. Adjusting the timeouts for elements does not resolve the flakiness.

[miherlosev]

No, we don’t need an additional information. We will update this thread as soon as we have news.

[kacy-weakley]

We have “skipJsErrors”: true in our configuration file.

This option hides the TestCafe issue. The tested web doesn’t have such an error. The described issue can be caused by this unhandled JavaScript error.

@miherlosev Ah, I see that in the error description now. Thank you for clarifying. In that case, what else do you need from me to move forward in resolving this first JavaScript error?