Memory problems and long load times for multiframe ultrasound data

Hello @dannyrb you seem most active on this problem, after several days of debugging I too am having issues loading Multiframe Ultrasound data. I have compiled a list of related issues from this repo at the bottom.

In my case I have RGB encoded, 400mb, 204 frame Ultrasound data in a single multiframe DCM file with transfer syntax 1.2.840.10008.1.2.1

Config:

var config = {
  maxWebWorkers: 1,
  startWebWorkersOnDemand: false,
  taskConfiguration: {
    decodeTask: {
      initializeCodecsOnStartup: false,
      loadCodecsOnStartup: false,
      usePDFJS: false,
      strict: false,
    },
  },
  webWorkerTaskPaths: [
    'https://unpkg.com/cornerstone-wado-image-loader@4.1.0/dist/610.bundle.min.worker.js',
    'https://unpkg.com/cornerstone-wado-image-loader@4.1.0/dist/888.bundle.min.worker.js',
  ],
};

A few observations:

• The per-frame load time (after the network call finishes) appears to scale with the size of the source file (dataset). I experience near-instant loads with a 12 frame file. Intermediate per-frame load times for a 100mb fluoroscopy file. ~1 sec / frame load times for the 400 mb file described above.
• If I throttle the decoding to the above configuration (load/initializeCodecs: false, maxWebWorkers:1) and only request one frame at a time before requesting for another (no queueing in webWorkerManager) then usually I can load all frames (this takes several minutes). Sometimes though the garbage collection seems to stop working and the memory will quickly climb to take up everything on my machine (16gb available, crashes around 15gb).
• After crashing, the memory will never get cleared
• Manually hitting the garbage collection button on chrome dev tools wipes this 15gb down to <1gb.
• running a profile on loading indicates that ~97% of the time is spent doing postMessage, seems like a lot of overheard for using webworkers.

Conclusions/Deductions from above observations

• For each frame request, a fully copy of the source dataset is being made
• The data is freed, but for some reason not being properly garbage collected, even after a very long time

What does a solution look like:

1. A fix to reduce memory allocation and time taken to load/decode a single frame
2. A bulk decode method that decodes all of the frames, even it is blocking.
3. A way to decode without the use of webworkers (and without allocating new memory)

Related issues

#235 #425 #411 https://github.com/cornerstonejs/cornerstoneWADOImageLoader/issues/156#issuecomment-794628050 https://github.com/cornerstonejs/cornerstoneWADOImageLoader/issues/428 #373

https://github.com/cornerstonejs/cornerstone/issues/576 https://github.com/cornerstonejs/cornerstone/issues/519

[dereklukacs]

@nyacoub

This seems to help fix the runaway memory problems, thanks!

[Ouwen]

Might need to use chrome memory profiler to take a look