Memory leak in loop with higher.innerloop_ctx!

I saw this problem was solved however, I still have it:

I am using the last version of higher 0.1.5, however still have a problem with memory leak:

with higher.innerloop_ctx(model, inner_opt, copy_initial_weights=False) as (fast_model, diffopt):
          for step, batch in enumerate(epoch_iterator):
                    report_memory()
                    print("-------------")
                    fast_model.train()
                    outputs = fast_model(batch)
                    loss = outputs[0]  
                    diffopt.step(loss)

memory (MB) | allocated: 678.73095703125 | max allocated: 678.73095703125 | cached: 734.0 | max cached: 734.0 memory (MB) | allocated: 1357.4619140625 | max allocated: 2036.19287109375 | cached: 2158.0 | max cached: 2158.0 memory (MB) | allocated: 12340.314453125 | max allocated: 13059.771484375 | cached: 13208.0 | max cached: 13208.0 memory (MB) | allocated: 23322.634765625 | max allocated: 24041.8564453125 | cached: 24190.0 | max cached: 24190.0

[nooralahzadeh]

@renesax14 There is no any version that mitigate this issue as I understood. It is on progress to solve !

[tathagatv]

@egrefen I’m facing this issue with the higher library when I am using the higher.path.make_functional function to make a forward pass through an LSTM model. How should I explicitly delete the backwards graph when I’m done with the inner and outer loop steps? As of now, the GPU consumption keeps increasing as more batches are processed.