Sugestion: add private repositories support in recipe engine

The following use case: As a server developer, I would like to supply the Github OAuth token to allow for private GitHub repos to be downloaded as part of the recipe.

There is a way to do it today directly via the command line. To also have it be exposed in a recipe.

git pull https://$OAUTH_TOKEN:[e-mail]/name/repo.git master

It could be entered in the UI and stored as a tmp variable that then can be defined in the yaml?

[tabarra]

for it to show on the UI, you have to create a var for it, so the var could have the token, or the var could be empty and the user fills it manually

[varnem2]

Digging up this issue, I’m going to share some thoughts here.

Background

As a review for the download_github task in the recipeEngine.js it seems as though it’s making 2 calls to github.com. (It seems this was refreshed from https://github.com/axios/axios over to https://github.com/sindresorhus/got between the time I’ve looked at this las.)

The first call occurs at https://github.com/tabarra/txAdmin/blob/63afc1048806d251669419fb6311ef506dbf672a/src/extras/recipeEngine.js#L113 where we get the default branch name for the repo to later use here https://github.com/tabarra/txAdmin/blob/63afc1048806d251669419fb6311ef506dbf672a/src/extras/recipeEngine.js#L122 .

Both of these calls would need authenticated through Github for the code to continue to work as it does today and expand it to a private repo.

Creating a personal access token in github as the instructions indicated here (We’ll define this token as GITHUB_PAT going forward.) allows a user to make both of these calls against an “internal” repo or a “private” repo. Upon further experimentation you’ll need to add a header to both of the indicated calls above following this object. { “key”: “Authorization”, “value”: Bearer ${GITHUB_PAT} }.

Passing in the header into the call where GITHUB_PAT equals an empty or a null value causes 401s to be returned from github.com regardless of the permission/visibility settings of the repo. The existence of the header does not limit the ability to download ‘public’ repos. The addition of this GITHUB_PAT token may cause tighter rate limiting than without any token at all, however that does still need to be experimented.

What I need help with is design and a few questions.

• What do you want the scope of this GITHUB_PAT to be? Recipe 1 or Recipe 2? Recipe 1 is an example of a global scope. While Recipe 2 would be an example of the scope being limited to the task?

Recipe 1

$engine: 3
$onesync: on
name: QBCore Framework RedM
version: 0.1.0
author: The QBCore Family
description: An advanced RedM framework including jobs, gangs, housing & more!
github_personal_access_token: ADSFADSADF

tasks:
  # Download Base Files
  - action: download_github
    src: https://github.com/qbcore-redm-framework/txAdminRecipe
    ref: main
    dest: ./tmp/qbrcore

Recipe 2

$engine: 3
$onesync: on
name: QBCore Framework RedM
version: 0.1.0
author: The QBCore Family
description: An advanced RedM framework including jobs, gangs, housing & more!

tasks:
  # Download Base Files
  - action: download_github
    src: https://github.com/qbcore-redm-framework/txAdminRecipe
    ref: main
    dest: ./tmp/qbrcore
    github_personal_access_token: ADSFADSADF

• Would we want to check the visibility of the repo to decide if we add the GITHUB_PAT token to the call? That would add a 3rd additional call to github.com for each github_download task.

• Would you want to do an implicit Auth where txAdmin would create a GITHUB_PAT for the user as the user uses the UI?

• Would you prefer a dark launch of the GITHUB_PAT in multiple PRs or just build out the whole thing?