bug in gradient accumulation example

https://github.com/huggingface/accelerate/blob/450d51ce0191020408bd3481bde85fe1dabaf289/examples/by_feature/gradient_accumulation.py#L160

Shouldn’t this condition be step % gradient_accumulation_steps != 0 because we want to avoid gradient averaging at each step other than gradient_accumulation_steps step?

[nilesh2797]

Let’s say gradient_accumulation_steps = 5, then In the current code optimizer.step() is getting called in step 1,2,3,4,6,7,8,9,11,... instead it should be getting called in step 0,5,10,15,20,... right? Basically it’s in the opposite logical block

[nilesh2797]

Thanks for the fix!