Restructuring feedstock/team update for better rate limit

Our current strategy with team updates is running into issues. ( https://github.com/conda-forge/conda-forge.github.io/issues/125 ) ( https://github.com/conda-forge/conda-forge.github.io/issues/88 ) That much is clear. Now how might we fix it. Here are some thoughts.

1. Manage staged-recipes with a different bot/token. Done with PR ( https://github.com/conda-forge/staged-recipes/pull/927 ).
   1. Easy to do.
   2. Doesn’t block more merges.
   3. Saves reviewer time (otherwise spent handling the fallout of rate limits).
   4. Small savings so isn’t sufficient alone.
   5. Could handle team additions and user additions. Done with PR ( https://github.com/conda-forge/staged-recipes/pull/733 )
   6. Obviates needed for triggering feedstock/team update script run. Also done with PR ( https://github.com/conda-forge/staged-recipes/pull/733 )
2. More bot accounts/tokens. Skipping per discussion with @pelson.
   1. Allows some scaling by number of bots.
   2. Effectively bumps our rate limit.
   3. Second bot is most expensive. The rest are cheap.
   4. Leaves us vulnerable to the same problems in the future.
3. Leverage the feedstock repo. ( related to @pelson’s suggestion )
   1. Already updating the feedstock repo.
   2. Only feedstocks that were added/changed get picked up. (couple order of magnitude improvement)
   3. Additional optimizations from diffing previous/current feedstock commits for name additions.
   4. Should actually catch everyone.
   5. Can always resort to the check and add everyone script.
4. Move to a time based non-CI (Heroku) approach as opposed to on demand. (self-throttling 😄) Done. See this ( https://github.com/conda-forge/conda-forge.github.io/issues/48#issuecomment-226571553 ).
   1. Saves time spent cloning everything. (cache)
   2. Reasonable for existing feedstocks.
   3. Can still be triggered as needed.
   4. Avoids script running when merging things here.
   5. Solves race condition problems. (debouncing requests)
   6. Kind of annoying for brand new feedstocks. (maintainers can’t fix problems right away)
5. Alternatives to a full Travis CI sync.
   1. This sync is probably costing us dearly with our GitHub API token.
   2. It probably needs to check the settings of each repo and update.
   3. This means it is O(N) on repos present here, which is bad when our rate limit is fixed.
   4. Need to be able to add a single repo to Travis CI. See issue ( https://github.com/travis-ci/travis-ci/issues/6320 ).
6. Move the feedstock/team update script off the webpage repo. ( https://github.com/conda-forge/conda-forge.github.io/issues/214 )
   1. No spurious runs when merging unrelated changes (e.g. minutes, pinnings, docs, etc.).

Basically we should do 1 to simplify the life of reviewers at staged-recipes. Whether or not it amounts to much savings for the rate limit is irrelevant as it saves people’s time. Examples of wasted time of people there include reporting these rate limiting issues, communicating rate limiting issues to each other, and working out the backlog at staged-recipes. Also, by doing this we would then break the problem into two pieces. How do we handle adding maintainers to a new feedstock vs. how do we add people to existing feedstocks? This would also put us on good footing for addressing 4.6.

Doing 2 is pretty easy. Though I’m not sure we should do it as it is a bit of a crutch.

Doing 3 has clear benefits as it is doubtful that many feedstocks are changing. With diffing the number that actually add maintainers, this should be quite small. We should definitely do this, but it will be a bit of work.

Doing 4 is generally a net plus. Though we should figure out how to address 4.6. For instance, 1.5 would help this. In general, this nicely couples with 1 and addresses the underlying reason for triggering the feedstock updating script on merge at staged-recipes.

Thoughts? Feedback other suggestions?

cc @pelson @ocefpaf @ericdill @janschulz

[jakirkham]

Going to close this for now. There is more work that can be done, but things seem to be more reliable now and less pressing.

[jjhelmus]

Would it make sense to sprinkle in more calls to print_rate_limiting_info(gh) in the create_feedstock.py script so that we can have a more fine grained view of where we are using the GitHub API? Perhaps before each subprocess.check_call?