screenshots saved in different folders when invoked from app vs. cli

Current behavior:

When using .screenshot() in a test from the command line with yarn cypress run -s cypress/integration/visual-regression/test-urls.js, screenshots are saved to cypress/screenshots/visual-regression/test-urls.js/. When using the same test from the app (cypress open), the screenshots are saved to cypress/screenshots/integration/visual-regression/test-urls.js/

Desired behavior:

Screenshots are saved to the same folders, no matter how the test is invoked

Versions

3.0.2, I guess this is introduced with #1826 / #1858

[chrisbreiding]

No, that’s definitely not expected. Looks like we didn’t take running all specs into account when implementing the new screenshot path rules. It’s a different issue than the original here, so I’m going to create a new GitHub issue for this.

[cangoektas]

I’m on 3.0.3 and I’m still experiencing this issue.

When running cypress run this is what my screenshots folder looks like:

cypress/screenshots
├── home.spec.js
│   └── home.png
└── login.spec.js
    └── login.png

When running cypress open and clicking on “Run all specs”, an “All specs” folder is created:

cypress/screenshots
├── All\ Specs
│   ├── home.png
│   └── login.png
├── home.spec.js
│   └── home.png
└── login.spec.js
    └── login.png

Is this expected?