[BUG] Playwright chromium ignores root CA certificates installed manually

Context:

• Playwright Version: 1.7.0
• Operating System: Ubuntu (PW docker image)
• Node.js version: v14.15.1
• Browser: Chromium 89.0.4344.0

Describe the bug I run our web portal in Docker/Kube and use self-signed certificate. Before running tests using PW docker image, I install root certificates into the Ubuntu in docker-compose file:

version: '3.7'
services:
  e2e:
    image: e2e_local_dev
    build:
      context: .
    volumes:
      - ./src:/e2e/src
      - ../.config/cert/ca.dev.crt:/usr/local/share/ca-certificates/ca.dev.crt
      - ../.config/cert/ca.local-dev.crt:/usr/local/share/ca-certificates/ca.local-dev.crt
      - ../.config/cert/ca.pal-root-dev.cer:/usr/local/share/ca-certificates/ca.pal-root-dev.crt
    environment:
      no_proxy: ${no_proxy},host.docker.internal
      BASE_URL: https://host.docker.internal:3001
      IS_DOCKER: 1
      DEBUG: pw:browser*
    command: /bin/sh -c "update-ca-certificates && yarn test:dev_parallel"

According to logs, certificates were installed:

Updating certificates in /etc/ssl/certs...
3 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

But when PW opens the portal URL, we can see SSL errors (-202) - CERT_AUTHORITY_INVALID:

2020-12-21 13:42:53.943 UTC [1028]:[1;36mOpening URL: https://host.docker.internal:3001.[0m
2020-12-21 13:42:53.943 UTC [1028]: api:info => page.goto started[0m
2020-12-21T13:42:54.123Z pw:browser [err] [1221/134254.123342:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21T13:42:54.340Z pw:browser [err] [1221/134254.340144:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21T13:42:54.347Z pw:browser [err] [1221/134254.342085:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21T13:42:54.365Z pw:browser [err] [1221/134254.361400:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21 13:42:56.18 UTC [1028]:[1;36mConsole: "[HMR] Waiting for update signal from WDS..."[0m
2020-12-21 13:42:56.65 UTC [1028]:[1;36mConsole: "%cDownload the React DevTools for a better development experience: https://fb.me/react-devtools"[0m
2020-12-21 13:42:56.65 UTC [1028]:[1;36mConsole: "font-weight:bold"[0m
2020-12-21 13:42:56.183 UTC [1028]:[1;36mConsole: "It looks like there are several instances of `@material-ui/styles` initialized in this application.\nThis may cause theme propagation issues, broken class names, specificity issues, and makes your application bigger without a good reason.\n\nSee https://material-ui.com/r/styles-instance-warning for more info."[0m
2020-12-21 13:42:56.210 UTC [1028]: api:info <= page.goto succeeded[0m

Root CA should be valid. Why Chromium ignores installed CA certificates? Did we install certs wrongly?

Full log:

docker-compose : Building e2e
At line:1 char:1
+ docker-compose -f docker-compose.local.dev.yaml up --build --abort-on ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Building e2e:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
Step 1/10 : FROM nexus.devops.bfsaws.net:7000/third-party/playwright:v1.7.0-bionic
 ---> c77f24164728
Step 2/10 : USER root
 ---> Using cache
 ---> 851ec825f1c1
Step 3/10 : ENV NODE_TLS_REJECT_UNAUTHORIZED=0
 ---> Using cache
 ---> 781209017a83
Step 4/10 : WORKDIR e2e
 ---> Using cache
 ---> 569e3c4a323d
Step 5/10 : RUN npm set strict-ssl false     && export NODE_OPTIONS=--max_old_space_size=4096     && yarn config set registry https://nexus.devops.bfsaws.net/repository/NPMPROXY
 ---> Using cache
 ---> ef4e54a78d0d
Step 6/10 : COPY package.json /e2e/
 ---> 49fb433ae6d9
Step 7/10 : COPY yarn.lock /e2e/
 ---> 11d093230fe6
Step 8/10 : COPY tsconfig.json /e2e/
 ---> e23ebc8e772f
Step 9/10 : COPY .env.dev-secrets /e2e/
 ---> 172db7d96f6d
Step 10/10 : RUN yarn
 ---> Running in 5fe2294f4e21
yarn install v1.22.10
[1/4] Resolving packages...
[91m(node:7) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use `node --trace-warnings ...` to show where the warning was created)
[0m[91mwarning @types/dotenv@8.2.0: This is a stub types definition. dotenv provides its own type definitions, so you do not need this installed.
[0m[91mwarning lighthouse > jsonld > request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
[0m[91mwarning lighthouse > jsonld > request > har-validator@5.1.5: this library is no longer supported
[0m[91mwarning mocha > debug@4.2.0: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
[0m[91mwarning mocha > chokidar > fsevents@2.1.3: Please update to v 2.2.x
[0m[2/4] Fetching packages...
info fsevents@2.1.3: The platform "linux" is incompatible with this module.
info "fsevents@2.1.3" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 124.75s.
Removing intermediate container 5fe2294f4e21
 ---> ee350957be85

Successfully built ee350957be85
Successfully tagged e2e_local_dev:latest
Creating e2e_e2e_1 ... 

[1A[2K
Creating e2e_e2e_1 ...[32mdone[0m
Attaching to e2e_e2e_1
Updating certificates in /etc/ssl/certs...
3 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
yarn run v1.22.10
$ mocha --reporter mocha-teamcity-reporter --timeout 90000 -r ts-node/register/transpile-only --require src/Generic/mochaHooks.ts src/Tests/**/Login.e2e.ts --grep '#full'
##teamcity[testSuiteStarted name='Login - Login page tests.' flowId='1028']
##teamcity[testStarted name='Login - Log into portal #full #build' captureStandardOutput='true' flowId='1028']
2020-12-21 13:42:52.354 UTC [1028]:[1;36mStarted test: Login - Log into portal #full #build[0m
2020-12-21 13:42:52.355 UTC [1028]:[1;36mBootstrap: NodeJS version v14.15.1[0m
2020-12-21 13:42:52.355 UTC [1028]:[1;36mBootstrap: Setting up global variables.[0m
2020-12-21 13:42:52.359 UTC [1028]:[1;36mBootstrap: Launching browser.[0m
2020-12-21 13:42:52.359 UTC [1028]:[1;36m{"timeout":60000,"args":["--start-maximized","--window-size=1920,1080","--no-sandbox","--disable-setuid-sandbox","--disable-dev-shm-usage","--enable-logging"],"logger":{}}[0m
2020-12-21 13:42:52.359 UTC [1028]: api:info => browserType.launch started[0m
2020-12-21T13:42:52.637Z pw:browser <launching> /root/.cache/ms-playwright/chromium-833159/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=TranslateUI,BlinkGenPropertyTrees,ImprovedCookieControls,SameSiteByDefaultCookies,LazyFrameLoading --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --user-data-dir=/tmp/playwright_chromiumdev_profile-8PSOC5 --remote-debugging-pipe --headless --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --start-maximized --window-size=1920,1080 --no-sandbox --disable-setuid-sandbox --disable-dev-shm-usage --enable-logging --no-startup-window
2020-12-21T13:42:52.647Z pw:browser <launched> pid=1105
2020-12-21T13:42:53.091Z pw:browser [err] [1221/134253.090288:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2020-12-21 13:42:53.151 UTC [1028]: api:info <= browserType.launch succeeded[0m
2020-12-21 13:42:53.152 UTC [1028]:[1;36mBootstrap: Browser launched. Browser version: 89.0.4344.0[0m
2020-12-21 13:42:53.153 UTC [1028]:[1;36mCreating new browser context.[0m
2020-12-21 13:42:53.153 UTC [1028]: api:info => browser.newContext started[0m
2020-12-21T13:42:53.172Z pw:browser [err] [1221/134253.171987:ERROR:vaapi_wrapper.cc(617)] Could not get a valid VA display
2020-12-21 13:42:53.196 UTC [1028]: api:info <= browser.newContext succeeded[0m
2020-12-21 13:42:53.197 UTC [1028]: api:info => browserContext.newPage started[0m
2020-12-21 13:42:53.530 UTC [1028]: api:info <= browserContext.newPage succeeded[0m
2020-12-21 13:42:53.531 UTC [1028]:[1;36mGeneric: Enabling logging of browser console errors and alerts.[0m
2020-12-21 13:42:53.531 UTC [1028]:[1;36mBootstrap: Logging into the Portal.[0m
2020-12-21 13:42:53.532 UTC [1028]:[1;36mBootstrap: NodeJS version v14.15.1[0m
2020-12-21 13:42:53.533 UTC [1028]:[1;36mBootstrap: Setting up global variables.[0m
2020-12-21 13:42:53.536 UTC [1028]:[1;36mBootstrap: Launching browser.[0m
2020-12-21 13:42:53.537 UTC [1028]:[1;36m{"timeout":60000,"args":["--start-maximized","--window-size=1920,1080","--no-sandbox","--disable-setuid-sandbox","--disable-dev-shm-usage","--enable-logging"],"logger":{}}[0m
2020-12-21 13:42:53.537 UTC [1028]: api:info => browserType.launch started[0m
2020-12-21T13:42:53.685Z pw:browser <launching> /root/.cache/ms-playwright/chromium-833159/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=TranslateUI,BlinkGenPropertyTrees,ImprovedCookieControls,SameSiteByDefaultCookies,LazyFrameLoading --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --user-data-dir=/tmp/playwright_chromiumdev_profile-oS85rJ --remote-debugging-pipe --headless --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --start-maximized --window-size=1920,1080 --no-sandbox --disable-setuid-sandbox --disable-dev-shm-usage --enable-logging --no-startup-window
2020-12-21T13:42:53.694Z pw:browser <launched> pid=1208
2020-12-21T13:42:53.770Z pw:browser [err] [1221/134253.770477:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2020-12-21 13:42:53.787 UTC [1028]: api:info <= browserType.launch succeeded[0m
2020-12-21 13:42:53.787 UTC [1028]:[1;36mBootstrap: Browser launched. Browser version: 89.0.4344.0[0m
2020-12-21 13:42:53.787 UTC [1028]:[1;36mCreating new browser context.[0m
2020-12-21 13:42:53.787 UTC [1028]: api:info => browser.newContext started[0m
2020-12-21T13:42:53.797Z pw:browser [err] [1221/134253.797485:ERROR:vaapi_wrapper.cc(617)] Could not get a valid VA display
2020-12-21 13:42:53.801 UTC [1028]: api:info <= browser.newContext succeeded[0m
2020-12-21 13:42:53.801 UTC [1028]: api:info => browserContext.newPage started[0m
2020-12-21 13:42:53.921 UTC [1028]: api:info <= browserContext.newPage succeeded[0m
2020-12-21 13:42:53.921 UTC [1028]:[1;36mGeneric: Enabling logging of browser console errors and alerts.[0m
2020-12-21 13:42:53.943 UTC [1028]:[1;36mOpening URL: https://host.docker.internal:3001.[0m
2020-12-21 13:42:53.943 UTC [1028]: api:info => page.goto started[0m
2020-12-21T13:42:54.123Z pw:browser [err] [1221/134254.123342:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21T13:42:54.340Z pw:browser [err] [1221/134254.340144:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21T13:42:54.347Z pw:browser [err] [1221/134254.342085:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21T13:42:54.365Z pw:browser [err] [1221/134254.361400:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21 13:42:56.18 UTC [1028]:[1;36mConsole: "[HMR] Waiting for update signal from WDS..."[0m
2020-12-21 13:42:56.65 UTC [1028]:[1;36mConsole: "%cDownload the React DevTools for a better development experience: https://fb.me/react-devtools"[0m
2020-12-21 13:42:56.65 UTC [1028]:[1;36mConsole: "font-weight:bold"[0m
2020-12-21 13:42:56.183 UTC [1028]:[1;36mConsole: "It looks like there are several instances of `@material-ui/styles` initialized in this application.\nThis may cause theme propagation issues, broken class names, specificity issues, and makes your application bigger without a good reason.\n\nSee https://material-ui.com/r/styles-instance-warning for more info."[0m
2020-12-21 13:42:56.210 UTC [1028]: api:info <= page.goto succeeded[0m
2020-12-21 13:42:56.210 UTC [1028]:[1;36mWaiting for login field selector.[0m
2020-12-21 13:42:56.211 UTC [1028]: api:info => page.waitForSelector started[0m
2020-12-21T13:42:57.049Z pw:browser [err] [1221/134257.048633:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21T13:42:58.091Z pw:browser [err] [1221/134258.091583:ERROR:cert_issuer_source_aia.cc(104)] AiaRequest::OnFetchCompleted got error -301
2020-12-21T13:42:58.097Z pw:browser [err] [1221/134258.095933:ERROR:ssl_client_socket_impl.cc(956)] handshake failed; returned -1, SSL error code 1, net_error -202
2020-12-21 13:42:58.497 UTC [1028]: api:info <= page.waitForSelector succeeded[0m

[implodingduck]

Apparently Chromium uses NSSDB for its certs rather than the OS store. The workaround that I did to get this to work is install certutil (on debian its libnss3-tools) and ran certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n root.implodingduck.local -i /usr/local/share/ca-certificates/extra/root.implodingduck.local.crt

Now when playwright chromium runs it can use the trusted root CA.

[pavelfeldman]

I’ll close this as a part of the bug triaging process. We have hundreds of bugs and feature requests with dozens and even hundreds of upvotes, while this one only has a few thumbs up. Please feel free to open a new bug and link this one if you’d like to see it addressed.