(Python packaging): Dependabot overrun trying to resolve all versions

General Issue

Dependabot is unable to determine most recent package versions, resulting in timeout

The Question

To reproduce:

1. Create a GitHub repo.
2. Install a few Python AWS CDK packages (17 in this case).
3. Configure Dependabot for pip.
4. Wait for Dependabot to do its thing.

What should happen: Dependabot should run successfully within a reasonable amount of time.

What happens: Dependabot did 664 HTTP GETs of https://pypi.org:443/pypi/aws-cdk.[…]/[…]/json) before giving up after 45 minutes of runtime. This seems to be the GitHub timeout for Dependabot jobs.

This means that users who have configured Dependabot on GitHub and are using Python AWS CDK packages are in danger of not getting automatic package update PRs, which is the sole purpose of Dependabot. Since Dependabot also fails silently (you have to go to https://github.com/$USER/$PROJECT/network/updates to see any indication of failure) this means projects are in danger of running vulnerable software.

I think this happens because of missing package metadata, resulting in Dependabot having to download every single version in order to determine which is the latest. But I don’t know wheels and other packaging features well enough to say for sure.

Workaround: Manually run poetry update after merging all the PRs Dependabot was able to create.

Untested workarounds:

• Lock dependency versions in pyproject.toml?
• Configure a separate job to run Dependabot? GitHub Actions should have a longer timeout than the built-in Dependabot configuration.

Other places this could be fixed:

• Dependabot could use the version number in the URL rather than fetching each version to (presumably) check its version number. I expect this wasn’t done because the URL doesn’t necessarily contain the full version number (as in the case of private repos).
• GitHub could bump their timeout.

CDK CLI Version

N/A

Framework Version

No response

Node.js Version

No response

OS

No response

Language

Python

Language Version

No response

Other information

No response

[l0b0]

Oh, I see. You’ve got aws-cdk 1.139, while the Geostore project still lingers at 1.92, presumably because of version incompatibilities with other packages. So this should be fixed once we’re able to upgrade somewhere between 1.93 and 1.139. We can probably close this; I’ll just create a new issue if this comes up with newer versions.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.