question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Document How To Install Python Package from a Private Repo

See original GitHub issue

I have a number of Python Packages in private (company) repos and I am using GitHub Actions to run pytest on commits. One of the repos depends on packages from other repos. When pip runs from the Action, I see the following error:

Collecting pyconfig@ git+ssh://git@github.com/sxi/pyconfig@master
Running command git clone -q 'ssh://****@github.com/sxi/pyconfig' /tmp/pip-install-wglwufhp/pyconfig
Cloning ssh://****@github.com/sxi/pyconfig (to revision master) to /tmp/pip-install-wglwufhp/pyconfig
Warning: Permanently added the RSA host key for IP address '140.82.114.4' to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
ERROR: Command errored out with exit status 128: git clone -q 'ssh://****@github.com/sxi/pyconfig' /tmp/pip-install-wglwufhp/pyconfig Check the logs for full command output.
##[error]Process completed with exit code 1.

Please document how the user can grant access to private repos to the Action. For example, I solved the problem using the following:

  1. For every Python package, create a step to check out the private repo
    - name: Checkout pyconfig from a private repo
      uses: actions/checkout@v2
      with:
        repository: <company>/pyconfig
        token: ${{ secrets.ACCESS_TOKEN }}
        path: pyconfig
  1. Modify the “Install dependencies” stop to pip install each Python package sourced from a private repo
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install /home/runner/work/<path>/pyconfig

While this works, it is a little tedious. Tech Support suggested I use HTTPS with a username and password to check out the packages from the private repositories. I would prefer to not use this method. It would likely require me to create and maintain a “fake” user account just for checking repositories in GitHub Actions. I would much prefer to use a personal access token (like I did above), but in a more simplified manner.

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:3
  • Comments:6

github_iconTop GitHub Comments

7reactions
ctothcommented, Aug 14, 2022

Is there a canonical way to do this now in 2022?

2reactions
ctothcommented, Sep 29, 2022

So far the best way I’ve found to do this is by creating a Machine User and inviting it to the organization.

Then, in requirements.txt you can add lines like:

git+ssh://git@github.com/<org>/<package>.git

Generate and add an SSH key for the machine user, then use something like

shimataro/ssh-key-action

to set the key in your workflow from a secret.

There is more information at Using organization Python package in Github actions without Python repository although this only seems to work for one package and so that’s why I ended up having to use the machine user approach as usually if you depend on one dependency in an organization you will depend on others.

Hope somebody finds this useful.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Installing Private Python Packages - Read the Docs
Read the Docs uses pip to install your Python packages. If you have private dependencies, you can install them from a private Git...
Read more >
Is it possible to use pip to install a package from a private ...
Run: pip install git+https://<PERSONAL ACCESS TOKEN>@github.com/<USERNAME>/<REPOSITORY>.git. Share.
Read more >
Create Your Custom, private Python Package That You Can ...
First, we'll focus on transforming your code into a python package so that people can easily install it. Then we'll put the package...
Read more >
Install Python Packages Hosted On Private GitHub
Learn how to install python packages through pip, when the packages are hosted within a private GitHub repository.
Read more >
How to Create a Private Python Package Repository - Linode
PyPI (Python Package Index) is a public repository of user-submitted packages that can be installed using pip install package .
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found