levit_128: Trying to backward through the graph a second time...

RuntimeError: Trying to backward through the graph a second time… (LOG)

This error may impact multiple models in this sweep, but only one repro is given below:

Repro cmd python benchmarks/dynamo/timm_models.py --accuracy --backend aot_eager --ddp --no-optimize-ddp --training --verbose --only levit_128 Log: https://gist.github.com/wconstab/1dfd032ed5cf4ff86b105aa4da6fa32c#file-timm_models_levit_128_dynamo_eager_ddp_no_optimize-log

Summary of Exception:

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

[anijain2305]

This model must have some unusual setup. Even pytorch native does not work. It will likely take some model-specific code in the benchmarks to make it work.

Is there any reason we want levit_128 to work? I propose to skip here.

[wconstab]

ah - --nothing is the feature i was wishing for. i was about to write my own version of it. thanks!