Memory leaks with detectAllFaces

Hi,

I have this coffeescript code snippet :

# Analyse the new frame
    analyseFrame: (next = _.noop) ->
    
        @testCount = 200 unless (typeof(@testCount) is 'number')
    
        # Skip if not capturing
        return unless Service.isCapturing
    
        # get frame
        _frame = Service.videoCapture.getFrame()
    
        # get frame date
        @currentFrameTime = Date.now()
    
        # clear old faces in history
        @refreshFaceHistory(@currentFrameTime)
    
        #convert frame to a tensor
        try
            _data = new Uint8Array(_frame.cvtColor(cv.COLOR_BGR2RGB).getData().buffer)
            _tensorFrame = tfjs.tensor3d(_data, [_frame.rows, _frame.cols, 3])
        catch _err
            @log.error "Error instantiating tensor !!!"
            @log.error _err.message

        faceapi.detectAllFaces(_tensorFrame, @faceDetectionOptions).then (_detectedFaces) =>
            @log.debug _detectedFaces
        
            # fill face history with detceted faces
            _detectedFaces = @fillFacesHistory(_detectedFaces)
        
            # draw boxes on image
            Service.videoCapture.drawFaceBoxes(_frame, _detectedFaces)
        
            # Get partial time
            Service.frameDuration = Date.now() - @currentFrameTime
        
            # write latency on image
            Service.videoCapture.writeLatency(_frame, Service.frameDuration)
        
            # show image
            Service.faceRecoUtils.showImage(_frame)
        
            # Call next
            _delayNextFrame = Math.max(0, 1000/@options.fps - Service.frameDuration)
        
            # DEBUG MEMORY DUMP every 30 sec  
            if @testCount > (5*30)
        
                @testCount = 0
  
                _dumpPath = path.resolve(__dirname + "./../memoryDump/" + @currentFrameTime + '.heapsnapshot')
                heapdump.writeSnapshot _dumpPath
   
        
        
            setTimeout =>
                # console.log "Next frame : #{_delayNextFrame}ms - TOTAL : #{_frameDuration}ms"
                @analyseFrame()
            , (_delayNextFrame)
        
        
        @testCount++

I feel like detectAllFaces is causing memory leaks. In deed, the corresponding nodejs process gains approximately 0.5 to 1% memory usage for each processed frame ! So my RAM get totally filled really quickly. I tried to get memory dump using heapdump but I don’t see anything anormal (all my dumps have same size on Google Chrome inspector). So this could means that memory leaks come from the C++ size of face-api.js, right ?

At first, I thought the leak could comes from combination of loops and promises or from Service.videoCapture. But I get similar results with the below code. The only difference being that RAM is filled slower. Which seems normal since the image used is smaller than my webcam’s frames.

# Analyse the new frame
    analyseFrame: (next = _.noop) ->

        # Skip if not capturing
        return unless Service.isCapturing

        _currentFunctionCallTime = Date.now()

        # get frame
        _frame = Service.videoCapture.getFrame()

        # if analyzer is ready, we can take a frame from webcam and process it
        if @analyzerIsReady

            @analyzerIsReady = false

            # get frame date
            @currentFrameTime =_currentFunctionCallTime

            # get frame
            _analyzedFrame = _frame

            # clear old faces in history
            @refreshFaceHistory(@currentFrameTime)

            @log.debug "faceHistory.length = ", Service.faceHistory.length

            _analyzedFrame = cv.imread('/home/loophole/eventbots-packages/eb/tiki-service-facedetection/models/bbt3.jpg')

            #convert frame to a tensor
            try
                _data = new Uint8Array(_analyzedFrame.cvtColor(cv.COLOR_BGR2RGB).getData().buffer)
                _tensorFrame = tfjs.tensor3d(_data, [_analyzedFrame.rows, _analyzedFrame.cols, 3])
            catch _err
                @log.error "Error instantiating tensor !!!"
                @log.error _err.message

            faceapi.detectAllFaces(_tensorFrame, @faceDetectionOptions).then (_detectedFaces) =>

                # fill face history with detceted faces
                _detectedFaces = @fillFacesHistory(_detectedFaces)

                # Get partial time
                Service.frameDuration = Date.now() - @currentFrameTime

                # Call next
                _delayNextFrame = Math.max(0, 1000/@options.fps - Service.frameDuration)

                setTimeout =>
                    @analyzerIsReady = true
                , (_delayNextFrame)

        _callDuration = Date.now() - _currentFunctionCallTime

        _delayNextCall = Math.max(0, 1000/@options.callPerSec - _callDuration)

        setTimeout =>
                @analyseFrame()
            , (_delayNextCall)

[justadudewhohacks]

Hey, there is no C++ side of face-api.js. If you are really experiencing memory leaks, it either comes from tensorflow or opencv4nodejs, which I find very unlikely as well since I didn’t experience any kind of leaks in long running applications using opencv4nodejs + face-api.js with similar code.

One thing that bothers me though, I don’t see you disposing the tensor _tensorFrame, which you create for every frame. If you do not dispose it after your done with it, e.g. _tensorFrame.dispose(), then your application will indeed leak memory. You can also check the tracked memory of tfjs using tf.memory().

[neumartin]

Solved!, only happend when run in debug mode, maybe never dispose because of that. Thanks!