question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unit tests should be fast 🐎

See original GitHub issue

This is a continuation of https://github.com/microsoft/vscode/issues/134972. A unit test shouldn’t take longer than 50ms. An easy way to “speed up” setTimeout and friends is @hediet’s runWithFakedTimers-utility

To reproduce these numbers

  • run ./scripts/test.sh --reporter json > ~/Desktop/test_out.json
  • open the file and remove the first non-json lines
  • run jq '.tests | map(select(.duration > 50)) | sort_by(.duration) | [.[] | "- [ ] " + (.duration|tostring) + "ms, " + .fullTitle ]' ~/Desktop/test_out.json

tests slower than 50ms

  • 51ms, ExtHostAuthentication createIfNone - false
  • 52ms, IndexDBStorageDatabase (browser) Clear
  • 54ms, MarkdownRenderer Code block renderer asyncRenderCallback should not be invoked if result is immediately disposed
  • 54ms, Notebook Find Update find matches basics
  • 56ms, Async Limiter async
  • 57ms, PFS writeFile (string) - flush on write
  • 57ms, ExtHostAuthentication Can get multiple sessions (with different scopes) in one extension
  • 57ms, ExtHostAuthentication Can get multiple sessions (from different providers) in one extension
  • 57ms, Notebook Find Update find matches basics 3
  • 57ms, Notebook Find Reset when match not found, #127198
  • 58ms, Notebook Find Update find matches, #112748
  • 60ms, FoldingRanges test max folding regions
  • 60ms, ExtHostAuthentication clearSessionPreference - true
  • 61ms, Notebook Find Update find matches basics 2
  • 69ms, SQLite Storage Library very large item value
  • 74ms, ExtHostAuthentication silently getting session should return a session (if any) regardless of preference - fixes #137819
  • 76ms, Workbench - TerminalWordLinkDetector should link words as defined by wordSeparators
  • 76ms, Files - TextFileEditorModel backup and restore (large, #121347)
  • 97ms, TerminalProfileService should get profiles from remoteTerminalService when there is a remote authority
  • 107ms, MarkdownRenderer Code block renderer asyncRenderCallback should not be invoked if dispose is called before code block is rendered
  • 121ms, IndexDBStorageDatabase (browser) Basics
  • 140ms, Native Modules (Windows, macOS) keytar
  • 157ms, Editor Controller issue #23913: Greater than 1000+ multi cursor typing replacement text appears inverted, lines begin to drop off selection
  • 201ms, Encoding autoGuessEncoding (UTF8)

Issue Analytics

  • State:open
  • Created a year ago
  • Reactions:2
  • Comments:21 (21 by maintainers)

github_iconTop GitHub Comments

2reactions
bpaserocommented, Jul 15, 2022

Working on a couple of things with https://github.com/microsoft/vscode/pull/155147

  • more adoption of runWithFakedTimers 🚀
  • figured out that we run a fs.fdatasyncSync(fd); for every writeFile to ensure data is flushed to disk which makes writing slower and figured out a way to disable this for most tests except the ones that want to test writing specifically
  • adopted inMemory scheme for some of my tests and even found a bug in our in-memory fs-provider

Before: scripts/test.sh 23.83s user 5.24s system 41% cpu 1:10.09 total

After: scripts/test.sh 24.31s user 5.06s system 47% cpu 1:02.10 total

2reactions
jriekencommented, Jun 9, 2022

@bpasero One thing we noticed is that quite some tests use the DiskFileSystemProvider. Those will surely benefit from an InMem-provider

Screenshot 2022-06-09 at 15 21 59
Read more comments on GitHub >

github_iconTop Results From Across the Web

Unit Tests Are FIRST: Fast, Isolated, Repeatable, Self ...
The faster your tests run, the more often you'll run them. Tests take half a minute to run? You'll maybe run them every...
Read more >
Just Say No to More End-to-End Tests - Google Testing Blog
Unit tests are fast. We only need to build a small unit to test it, and the tests also tend to be rather...
Read more >
What are the pros and cons of automated Unit Tests vs ...
2) The second value of unit tests is that if they are properly written they are very very fast. If I make a...
Read more >
asp.net - Are unit tests really that useful?
And once you have automated tests, unit testing makes a lot more sense, since it provides a faster turnaround than big integration or...
Read more >
FIRST Principles for Writing Good Unit Tests - HowToDoInJava
Unit tests should be fast otherwise they will slow down your development/deployment time and will take longer time to pass or fail.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found