shallow_since appears to be broken

Description of the problem / feature request:

shallow_since appears to be broken.

Bugs: what’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Try to build //:BUILD.boost from https://github.com/nelhage/rules_boost/commit/df908358c605a7d5b8bbacde07afbaede5ac12cf. You’ll receive a recommendation for shallow_since. Insert that recommendation, and suddenly Bazel fails to find that commit at all.

What operating system are you running Bazel on?

Windows

What’s the output of bazel info release?

release 1.1.0

What’s the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

Create an empty BUILD.bazel alongside the following WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(name="rules_boost", commit="df908358c605a7d5b8bbacde07afbaede5ac12cf", remote="https://github.com/nelhage/rules_boost")

Now run bazel --ignore_all_rc_files query "deps(@rules_boost//:BUILD.boost)":

bazel --ignore_all_rc_files query "deps(@rules_boost//:BUILD.boost)"
@bazel --ignore_all_rc_files clean --expunge

It outputs something like the following (after formatting/redactions):

INFO: Writing tracer profile to 'command.profile.gz'
DEBUG: Rule 'rules_boost' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1569511515 +0200"
DEBUG: Call stack for the definition of repository 'rules_boost' which is a git_repository (rule definition at external/bazel_tools/tools/build_defs/repo/git.bzl:195:18):
 - WORKSPACE:2:1
ERROR: no such target '@rules_boost//:BUILD.boost': target 'BUILD.boost' not declared in package ''; however, a source file of this name exists.  (Perhaps add 'exports_files(["BUILD.boost"])' to /BUILD?) defined by external/rules_boost/BUILD
Loading: 1 packages loaded

Now add shallow_since="1569511515 +0200" as recommended and re-run the query:

INFO: Writing tracer profile to 'command.profile.gz'
INFO: Call stack for the definition of repository 'rules_boost' which is a git_repository (rule definition at external/bazel_tools/tools/build_defs/repo/git.bzl:195:18):
 - WORKSPACE:2:1
ERROR: An error occurred during the fetch of repository 'rules_boost':
error running 'git reset --hard df908358c605a7d5b8bbacde07afbaede5ac12cf' while working with @rules_boost:
fatal: Could not parse object 'df908358c605a7d5b8bbacde07afbaede5ac12cf'.
ERROR: no such package '@rules_boost//'
Loading: 0 packages loaded

Suddenly Bazel seems to be unable to find the commit altogether.

[nelhage]

I did some digging here, and reported an upstream bug to git

The upshot is that --shallow-since is subtly incorrect when the provided timestamp would cause us to include some parents of a merge, but not others. In the case provided here, nelhage/rules_boost@df90835 has a commit timestamp of 1569511515. That commit is the second part of a merge, nelhage/rules_boost@76f3064; that merge’s first parent is nelhage/rules_boost@a83197e, with an older commit timestamp of 1569487118

The bug I reported results in git discarding all parents of the merge (nelhage/rules_boost@76f3064), including the desired commit.

For now, the workaround for bazel users – where available – is to switch your commit= line to point to a commit on the first-parent spine, e.g. nelhage/rules_boost@76f3064. If you add the suggested shallow_since line for that commit, things work fine.

For Bazel itself, it might be nice to recognize this situation and adjust the shallow_since recommendation, at least until the upstream bug can be sorted out. I’m not certain offhand what the right/easiest heuristic to detect it is, though.

[asuffield]

It turns out that the workaround proposed here doesn’t work either.

The shallow_since bug is even worse than I expected. It stops at the first commit which has any parents before the cutoff time. This means we have the following horrible scenario:

1. Latest commit on main branch is aaaa, with parents baaa and bbbb. aaaa is at 8am, baaa is at 6am and bbbb is at 4am
2. I create a git_repository rule with commit = “aaaa”, shallow_since = “4am”
3. This works fine when I build it
4. Somebody adds a new commit caaa to main branch, with parents aaaa and cbbb. caaa is at 9am, cbbb is at 2am
5. git clone --shallow_since=4am will now fetch only caaa because it has a parent before 4am, and aaaa will not be fetched

This is completely unusable and I don’t see a workaround.