question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

TorchVision Main branch fails with `slow_conv2d: grad_weight must be contiguous`

See original GitHub issue

🐛 Describe the bug

It looks like a breakage caused by the 20220512 PyTorch Core Nightly.

Several CI jobs fail with:

test_classification_model[cpu-convnext_base]
Traceback (most recent call last):
  File "/root/project/test/test_models.py", line 628, in test_classification_model
    _check_input_backprop(model, x)
  File "/root/project/test/test_models.py", line 181, in _check_input_backprop
    out[0].sum().backward()
  File "/root/project/env/lib/python3.10/site-packages/torch/_tensor.py", line 399, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/root/project/env/lib/python3.10/site-packages/torch/autograd/__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: slow_conv2d: grad_weight must be contiguous

So far I’ve identified the following PRs as potential causes:

Given that this network uses LayerNorm changes from the following PRs are also like to have caused it:

cc @mingfeima @VitalyFedyunin

Versions

Latest main branch 03bb324576d9fe86a15a7b86a43638a838234fbd

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:1
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
mingfeimacommented, May 12, 2022

i will check if i need to add more test case in pytorch (the old test cases failed to find out the code flaw)

0reactions
datumboxcommented, May 14, 2022

Fixed on latest nightly

Read more comments on GitHub >

github_iconTop Results From Across the Web

No results found

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found