[Windows] MSS is not thread-safe

The problem

Hello, I try to do stuff like this:

• Http request from JS
• Python handles it with Flask
• When Python Flask gets a request it will grab a current display view (screenshot), resize it, convert to base64 and return it as response

It’s all ok, but when I send second http request - python mss fails.

(there’s .js and .py files because it could somehow help)

server.py

# Image processing, FPS in console
import mss, cv2, base64, time
import numpy as np
from PIL import Image as i

# Get current frame of second monitor
def getFrame():
    start_time = time.perf_counter()

    # Get frame (only rgb - smaller size)
    frame_rgb     = mss.mss().grab(mss.mss().monitors[2]).rgb # type: bytes, len: 1280*720*3 (w, h, r, g, b)

    # Convert it from bytes to resize
    frame_image   = i.frombytes("RGB", (1280, 720), frame_rgb, "raw", "RGB") # PIL.Image.Image
    frame_array   = np.array(frame_image) # type: numpy.ndarray
    frame_resized = cv2.resize(frame_array, (640, 360), interpolation = cv2.INTER_CUBIC) # type: numpy.ndarray

    # Encode to base64 - prepared to send
    frame_base64  = base64.b64encode(frame_resized) # type: bytes, len: 640*360*4 (w, h, r, g, b, ???)

    print(f'{ round( 1 / (time.perf_counter() - start_time), 2) } fps')
    return frame_base64

# Flask request handler
from flask import Flask, request
from flask_cors import CORS

app = Flask(__name__)
cors = CORS(app)

@app.route('/frame_base64')
def frame_base64():
    return getFrame()

app.run(debug=True, port=7999)

script.js (little weird because of compilation from coffeescript)

(function() {

  $(document).ready(function() {

    return $.ajax({
      type: 'get',
      url: ' http://127.0.0.1:7999/frame_base64',
      success: (response) => {
        return console.log(response);
      }
    });

  });

}).call(this);

Full console log:

D:\web\projects\html-display-stream\backend>python server.py
 * Serving Flask app "server" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 258-577-249
 * Running on http://127.0.0.1:7999/ (Press CTRL+C to quit)
18.0 fps
127.0.0.1 - - [26/Feb/2020 00:36:05] "GET /frame_base64 HTTP/1.1" 200 -
127.0.0.1 - - [26/Feb/2020 00:36:06] "GET /frame_base64 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 2463, in __call__

    return self.wsgi_app(environ, start_response)
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 2449, in wsgi_app

    response = self.handle_exception(e)
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask_cors\extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1866, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\_compat.py", line 39, in reraise
    raise value
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 2446, in wsgi_app

    response = self.full_dispatch_request()
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask_cors\extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\_compat.py", line 39, in reraise
    raise value
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "D:\web\projects\html-display-stream\backend\server.py", line 33, in frame_base64
    return getFrame()
  File "D:\web\projects\html-display-stream\backend\server.py", line 11, in getFrame
    frame_rgb     = mss.mss().grab(mss.mss().monitors[2]).rgb # type: bytes, len: 1280*720*3 (w, h, r, g, b)
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python37\lib\site-packages\mss\windows.py", line 291, in grab
    raise ScreenShotError("gdi32.GetDIBits() failed.")
mss.exception.ScreenShotError: gdi32.GetDIBits() failed.

Additional info:

Windows 7 x64 Service Pack 1 Monitors: 1920×1080, 1280×720 Python 3.7.6 pip 20.0.2 python-mss last version of now

P.S.: Sorry for my bad English.

Thanks

[narumi147]

During the performance test, I find that this bug affects not only srcdc and bmp Overridden problems. If run in a large loop without any time.sleep(), bmp/srcdc/memdc(their windows object) will be written by multiple threads at same time, and unpredictable error occurred then raise gdi32.GetDIBits() failed. So In the following performance tests, I add a lock in the origin MSS class and acquire it inside grab method, just the same as I mentioned above.

Here we take total 1000 full size screenshots of two monitors through 1/10/100 threads. No significant performace gap investigated.

With old mss
1000 shots *   1 threads: total 50.170 seconds, 19.932 fps 
 100 shots *  10 threads: total 50.147 seconds, 19.941 fps
  10 shots * 100 threads: total 50.282 seconds, 19.888 fps
With new mss
1000 shots *   1 threads: total 50.179 seconds, 19.929 fps
 100 shots *  10 threads: total 50.015 seconds, 19.994 fps
  10 shots * 100 threads: total 50.196 seconds, 19.922 fps

[narumi147]

If you mean the tests of regression_issue_128 and regression_issue_135, these tests all success both in main thread and child thread.