cypress e2e testing does not exit on error in development server

Version

3.4.0

Environment info

Environment Info:

  System:
    OS: macOS
  Binaries:
    Node: 11.6.0 - /usr/local/bin/node
    Yarn: Not Found
    npm: 6.7.0 - /usr/local/bin/npm
  npmPackages:
    @ascendancyy/vue-cli-plugin-stylelint: ^1.1.0 => 1.1.0 
    @fortawesome/vue-fontawesome: ^0.1.3 => 0.1.5 
    @vue/babel-helper-vue-jsx-merge-props:  1.0.0-beta.2 
    @vue/babel-plugin-transform-vue-jsx:  1.0.0-beta.2 
    @vue/babel-preset-app:  3.3.0 
    @vue/babel-preset-jsx:  1.0.0-beta.2 
    @vue/babel-sugar-functional-vue:  1.0.0-beta.2 
    @vue/babel-sugar-inject-h:  1.0.0-beta.2 
    @vue/babel-sugar-v-model:  1.0.0-beta.2 
    @vue/babel-sugar-v-on:  1.0.0-beta.2 
    @vue/cli-overlay:  3.3.0 
    @vue/cli-plugin-babel: ^3.2.0 => 3.3.0 
    @vue/cli-plugin-e2e-cypress: ^3.2.0 => 3.3.0 
    @vue/cli-plugin-eslint: ^3.2.1 => 3.3.0 
    @vue/cli-plugin-unit-mocha: ^3.2.0 => 3.3.0 
    @vue/cli-service: ^3.2.2 => 3.3.0 
    @vue/cli-shared-utils:  3.3.0 
    @vue/component-compiler-utils:  2.5.0 
    @vue/eslint-config-standard: ^4.0.0 => 4.0.0 
    @vue/preload-webpack-plugin:  1.1.0 
    @vue/test-utils: ^1.0.0-beta.27 => 1.0.0-beta.28 
    @vue/web-component-wrapper:  1.2.0 
    babel-helper-vue-jsx-merge-props:  2.0.3 
    babel-plugin-transform-vue-jsx:  4.0.1 
    eslint-plugin-vue: ^5.0.0 => 5.1.0 
    vue: ^2.5.21 => 2.5.22 
    vue-cli-plugin-element: ^1.0.0 => 1.0.1 
    vue-cli-webpack:  1.0.0 
    vue-eslint-parser:  2.0.3 
    vue-hot-reload-api:  2.3.1 
    vue-i18n: ^8.5.0 => 8.7.0 
    vue-loader:  15.5.1 
    vue-router: ^3.0.2 => 3.0.2 
    vue-style-loader:  4.1.2 
    vue-template-compiler: ^2.5.21 => 2.5.22 
    vue-template-es2015-compiler:  1.8.1 
    vue-test-chai: ^1.0.0-3 => 1.0.0-3 
    vuex: ^3.0.1 => 3.1.0 
  npmGlobalPackages:
    @vue/cli: 3.4.0

Steps to reproduce

1. Have cli-plugin-e2e-cypress installed in your project and configure a CI to run tests
2. Produce an error in one of your components which would break the build
3. Run e2e tests

What is expected?

When end-to-end tests are run inside a CI it should exit with an error code when there was an error building the app.

This way the CI fails and does not time out.

What is actually happening?

The WebpackDevServer shows errors, Cypress never starts and the CI might fail after a given timeout.

---

The cli-plugin-e2e-cypress waits for the serve command to finish before starting up cypress.

If there’s an error in the build, the serve command never finishes, as the WebpackDevServer waits for changes to rebuild the app. However this will never happen in a CI and it might time out.

[Seraf]

Hello, we also have this problem sometimes on our CI testing the PR which hangs the build. Is there some work or someone working on this?

Thanks

[MatthiasGrandl]

Related, but shouldn’t the process also exit, when all specs ran successfully? It just keeps running otherwise which isn’t ideal. I think this should happen if the --headless parameter is passed.