Add a pre-commit check for secrets

Is your feature request related to a specific problem?

We want to avoid accidentally committing secrets to the repo

Describe the solution you’d like

Add a pre-commit task that scans for secrets. We should additionally leverage the noxfile to verify that developers have this enabled.

Describe alternatives you’ve considered, if any

Not sure what other options we have besides pre-commits here, or what might be better

[ThomasLaPiana]

@daveqnet I’m enjoying this thread and your findings! Thanks for putting in this work

[PSalant726]

Here are the tools that I turned up when looking into this a few months ago:

• trufflehog (OSS)
• gitleaks (OSS)
• gitguardian / ggshield (commercial, but free for OSS projects)
• yelp/detect-secrets (OSS)

Gitleaks is the one that I happen to be the most familiar with. It works well, but I don’t feel strongly about the tool that we integrate.

I like GitGuardian, if only because their documentation is good e.g. blog.gitguardian.com/setting-up-a-pre-commit-git-hook-with-gitguardian-shield-to-scan-for-secrets

If it’s only free for OSS, then keep in mind that we also have private repos where we’ll be developing connectors and other tools that integrate 3rd party APIs. Ideally we don’t need to integrate multiple tools.

• I think secrets detection hooks should be executed pre-push, not pre-commit.

I agree with Paul - a pre-commit hook is where we should enforce things. It’s true that it would be less “noisy” on pre-push, but if you only learn about your mistake after it’s already made, then you have the cognitive load of dealing with cleanup. The pre-commit hook is intended to be used to prevent those mistakes from being made at all, and the check should pass for 99% of commits anyway.

• @PSalant726 has some strong opinions on native git hooks vs python scripts (he’ll be able to explain better than I can).

The pre-commit framework uses native git hooks, it just makes them easy to setup and maintain.

I wouldn’t call my opinion on this strong, but I prefer to avoid adding dependencies whenever possible. I’ve never had an issue setting up native git hooks, so the dependency feels unnecessary here.