Destructive operation on submodules with `submodule.recurse = true` git setting

describe your issue

I added pre-commit to an ansible playbook repository that does extensive usage of git submodules for roles.

I had two submodules with extensive changes:

$ gst
On branch master
Your branch is ahead of 'origin/master' by 3 commits.
  (use "git push" to publish your local commits)
...
        modified:   roles/dnscrypt-proxy (modified content) <-- submodule with extensive (non-commited) changes
        modified:   roles/syncthing (modified content) <-- another submodule with extensive (non-commited) changes
...

Upon doing a commit I received an error (check ~/.cache/pre-commit/pre-commit.log section) due to having some problems in my repo and its submodules. Notice that I cold not even run git status afterwards:

$ gst
fatal: not a git repository: roles/matomo/../../.git/modules/roles/matomo

After fixing the problems:

$ rm -rf roles/matomo/ .git/modules/roles/matomo/
$ git submodule update
Cloning into '/home/apoc/w/posg-ops/roles/matomo'...
Submodule path 'roles/matomo': checked out '######'
fatal: could not get a repository handle for submodule 'roles/php'
$ rm -rf roles/php/ .git/modules/roles/php
$ git submodule update
Cloning into '/home/apoc/w/posg-ops/roles/php'...
Submodule path 'roles/php': checked out '######'

and running git apply to have my changes back:

$ git apply /home/a666/.cache/pre-commit/patch1632709290-1076629

I noticed that my two submodules now had no changes in them… And unfortunately I see nothing in pre-commit cache area to help me recover them.

pre-commit --version

pre-commit 2.15.0

.pre-commit-config.yaml

---
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.0.1
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files

  - repo: https://github.com/editorconfig-checker/editorconfig-checker.python
    rev: 2.3.54
    hooks:
      - id: editorconfig-checker

  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.51.0
    hooks:
      - id: terraform_validate
      - id: terraform_fmt
      - id: checkov
      - id: terrascan
      - id: terraform_tfsec

~/.cache/pre-commit/pre-commit.log (if present)

version information

pre-commit version: 2.15.0
sys.version:
    3.9.7 (default, Aug 31 2021, 13:28:12)
    [GCC 11.1.0]
sys.executable: /home/a666/.local/pipx/venvs/pre-commit/bin/python
os.name: posix
sys.platform: linux

error information

An unexpected error has occurred: CalledProcessError: command: ('/usr/lib/git-core/git', 'checkout', '--', '.')
return code: 255
expected return code: 0
stdout: (none)
stderr:
    Migrating git directory of 'roles/fail2ban' from
    '/home/a666/ops/roles/fail2ban/.git' to
    '/home/a666/ops/.git/modules/roles/fail2ban'
    fatal: not a git repository: ../../.git/modules/roles/matomo
    error: Submodule 'roles/matomo' could not be updated.
    Migrating git directory of 'roles/openproject' from
    '/home/a666/ops/roles/openproject/.git' to
    '/home/a666/ops/.git/modules/roles/openproject'
    Migrating git directory of 'roles/oryhydra' from
    '/home/a666/ops/roles/oryhydra/.git' to
    '/home/a666/ops/.git/modules/roles/oryhydra'
    fatal: not a git repository: ../../.git/modules/roles/php
    error: Submodule 'roles/php' could not be updated.

Traceback (most recent call last):
  File "/home/a666/.local/pipx/venvs/pre-commit/lib/python3.9/site-packages/pre_commit/error_handler.py", line 65, in error_handler
    yield
  File "/home/a666/.local/pipx/venvs/pre-commit/lib/python3.9/site-packages/pre_commit/main.py", line 368, in main
    return hook_impl(
  File "/home/a666/.local/pipx/venvs/pre-commit/lib/python3.9/site-packages/pre_commit/commands/hook_impl.py", line 237, in hook_impl
    return retv | run(config, store, ns)
  File "/home/a666/.local/pipx/venvs/pre-commit/lib/python3.9/site-packages/pre_commit/commands/run.py", line 399, in run
    exit_stack.enter_context(staged_files_only(store.directory))
  File "/usr/lib/python3.9/contextlib.py", line 448, in enter_context
    result = _cm_type.__enter__(cm)
  File "/usr/lib/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/home/a666/.local/pipx/venvs/pre-commit/lib/python3.9/site-packages/pre_commit/staged_files_only.py", line 92, in staged_files_only
    with _intent_to_add_cleared(), _unstaged_changes_cleared(patch_dir):
  File "/usr/lib/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/home/a666/.local/pipx/venvs/pre-commit/lib/python3.9/site-packages/pre_commit/staged_files_only.py", line 61, in _unstaged_changes_cleared
    cmd_output_b('git', 'checkout', '--', '.', env=no_checkout_env)
  File "/home/a666/.local/pipx/venvs/pre-commit/lib/python3.9/site-packages/pre_commit/util.py", line 154, in cmd_output_b
    raise CalledProcessError(returncode, cmd, retcode, stdout_b, stderr_b)
pre_commit.util.CalledProcessError: command: ('/usr/lib/git-core/git', 'checkout', '--', '.')
return code: 255
expected return code: 0
stdout: (none)
stderr:
    Migrating git directory of 'roles/fail2ban' from
    '/home/a666/ops/roles/fail2ban/.git' to
    '/home/a666/ops/.git/modules/roles/fail2ban'
    fatal: not a git repository: ../../.git/modules/roles/matomo
    error: Submodule 'roles/matomo' could not be updated.
    Migrating git directory of 'roles/openproject' from
    '/home/a666/ops/roles/openproject/.git' to
    '/home/a666/ops/.git/modules/roles/openproject'
    Migrating git directory of 'roles/oryhydra' from
    '/home/a666/ops/roles/oryhydra/.git' to
    '/home/a666/ops/.git/modules/roles/oryhydra'
    fatal: not a git repository: ../../.git/modules/roles/php
    error: Submodule 'roles/php' could not be updated.

[a666]

Well, submodules were always like a weird git workaround to a difficult problem (I think) and not very though out…

[asottile]

submodule.recurse = 1 makes git checkout -- . also run git checkout -- . in each submodule – seems like a really bad git feature