Suspected memory leak

Hi, thank you for sharing your python wrapper for darknet. This is very useful to us.

We are currently doing a bench-marking experiment using your library, which includes setting different input image sizes in the cfg file and finding the performance in terms of speed.

However, we seem to be observing that as we repeatedly load a new network (with a new cfg) in a for loop, the GPU memory usage steadily increase each round, even as we went from a larger input image size to a smaller one.

Here’s our test script for reference:

from io import BytesIO
import numpy as np
import requests
import cv2
import time

from pydarknet import Detector, Image
from check_input_size import check_input_size_range
from set_input_size import set_input_size

im_path = 'frame0001.jpg'
img = cv2.imread(im_path)
img = Image(img)
cfg = 'yolo/cfg/yolov3.cfg'
data_file = 'yolo/cfg/coco.data'
wts = 'yolo/weights/yolov3.weights'

min_size = 40
max_size = 1000
sizes, _ = check_input_size_range(cfg, min_size, max_size)
print('Will run for {} sizes'.format(len(sizes)))
sizes = list(reversed(sizes))

repeat = 100
for size in sizes:
    set_input_size(cfg=cfg, new_cfg=cfg, size=size)
    net = Detector(bytes("yolo/cfg/yolov3.cfg", encoding="utf-8"), 
                   bytes("yolo/weights/yolov3.weights", encoding="utf-8"), 
                   1,
                   bytes("yolo/cfg/coco.data", encoding="utf-8"))
    duration = 0
    img = cv2.imread(im_path)
    img = Image(img)
    for i in range(repeat):
        print('Repeating {: >4}th time for size {: <4}'.format(i+1, size), end='\r')
        tic = time.time()
        results = net.detect(img)
        toc = time.time()
        duration = ( (toc-tic) + i * duration )/(i+1)
    with open('temp.log','a') as f:
        f.write('{}:{}\n'.format(size, duration))
    net = None
    img = None
    results = None

set_input_size is a simple function that goes into the cfg file to change the size of the input image specified there for varying sizes that are valid.

As we loop through the for loop, we observe the GPU memory usage slowly rising by around 20+MB each round. We added print statement in the .pyx file to check that the network and image variables are getting freed and they are correctly freed at the end of each round. Therefore we are not sure if its something else whose memory is not getting freed.

This is not a big issue now, but just concerned if the memory leak might be a problem in the future. Any ideas? Thank you!

[madhawav]

Hi, Thanks a lot @DinoHub I will look into this.

Can you try manually deleting the network after each iteration? Modify the last bit of your code as follows.

    net = None
    img = None
    results = None
    del net

This would enforce the destructor of Detector to be called immediately, which should notify darknet to release the model. Otherwise, the destructor may not be called until Python Garbage Collector kicks in.

[madhawav]

Hi, Thank you for your reply. Closing the issue.