question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Save bandwidth using repo mirror

See original GitHub issue

As I mentioned on #278, I created mirrors of the deb and rpm repositories in the hopes that we can reduce the bandwidth requirements on packagecloud without giving shiftkey any more work by disrupting his CI workflow. To this end, I have setup a complete mirror. The mirror has the following properties/features:

  • IPv4/IPv6 support
  • http/https support
  • Syncs with packagecloud every 6 hours
  • Works with both deb and rpm based distributions
  • Is located in New Jersey near NYC, USA
  • Preserves shiftkey’s GPG signature; so I cannot tamper with the files
  • Remains functional (though stale) if packagecloud runs our of bandwidth or goes down
  • Highly scalable, static ngnix server

I encourage anyone reasonably close to the node (i.e. US/Europe) to test this mirror as a way of reducing bandwidth on packagecloud. This way, we can hopefully keep the packagecloud repo up all month and eventually reduce shiftkey’s hosting costs. The original packagecloud server is probably faster in most locations, but how much does this matter for a repo with just one relatively small package?

Usage

Debian/Ubuntu

The mirror url is https://mirror.mwt.me/ghd/deb/. You can change this in your sources manually, or by running the following modified command from the readme:

$ wget -qO - https://keys.openpgp.org/vks/v1/by-keyid/7650C43BBA3C3A2C | sudo tee /etc/apt/trusted.gpg.d/shiftkey-desktop.asc > /dev/null
$ sudo sh -c 'echo "deb [arch=amd64] https://mirror.mwt.me/ghd/deb any main" > /etc/apt/sources.list.d/packagecloud-shiftkey-desktop.list'
$ sudo apt update

Red Hat/CentOS/Fedora

The mirror url is https://mirror.mwt.me/ghd/rpm/. You can change this in your sources manually, or by running the following modified command from the readme:

$ sudo rpm --import https://keys.openpgp.org/vks/v1/by-keyid/7650C43BBA3C3A2C
$ sudo sh -c 'echo -e "[shiftkey]\nname=GitHub Desktop\nbaseurl=https://mirror.mwt.me/ghd/rpm\nenabled=1\ngpgcheck=0\nrepo_gpgcheck=1\ngpgkey=https://packagecloud.io/shiftkey/desktop/gpgkey" > /etc/yum.repos.d/shiftkey-desktop.repo'

OpenSUSE distribution

The mirror url is https://mirror.mwt.me/ghd/rpm/. You can change this in your sources manually, or by running the following modified command from the readme:

$ sudo rpm --import https://keys.openpgp.org/vks/v1/by-keyid/7650C43BBA3C3A2C
$ sudo sh -c 'echo -e "[shiftkey]\nname=GitHub Desktop\nbaseurl=https://mirror.mwt.me/ghd/rpm\nenabled=1\ngpgcheck=0\nrepo_gpgcheck=1\ngpgkey=https://packagecloud.io/shiftkey/desktop/gpgkey" > /etc/zypp/repos.d/shiftkey-desktop.repo'

Notes

  • You can also use the mirror over Cloudflare by replacing mirror.mwt.me with mirror-cf.mwt.me in any of the steps above.
  • I have also mirrored the gpg key to https://mirror.mwt.me/ghd/gpgkey though I don’t suggest anyone use it, as it defeats the purpose of repo signing.
  • I have not thoroughly tested the rpm repo because I do not use or have any experience with rpm-based distros. Please let me know below if it does/doesn’t work for you. I use the deb one myself.
  • This solution is flexible in the sense that the mirror can be pointed at any other provider that shiftkey may choose to move to in the future.
  • Feel free to make mirrors that clone from my mirror. However, my rpm repo workflow is pretty hackish. If you are using an rpm distro, it’s probably better for me to clone from you rather than the other way around.
  • I’m happy to make this the default repo in the readme instructions to help reduce bandwidth from new users.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:7

github_iconTop GitHub Comments

1reaction
crazyn2commented, Feb 17, 2022

Thanks

1reaction
crazyn2commented, Feb 16, 2022

Sorry, after follow the modified steps, I got those errors in Ubuntu20.04 amd64

Err:10 https://mirror.mwt.me/ghd/deb any InRelease                             
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B7E65990E879472F`
Read more comments on GitHub >

github_iconTop Results From Across the Web

Mirror repositories to save bandwidth - Linux Video Tutorial
One solution to this is to set up a caching proxy that your clients use, which will store a copy of packages that...
Read more >
Save Bandwidth by Setting Up a Fedora Mirror
The most essential requirement for mirroring is bandwidth. How long your download will take depends on the available bandwidth. Mirroring over a ...
Read more >
How to limit network bandwidth apt-mirror
Just add the following line of code to your mirror.list file: set limit_rate 50k. The answer also states that the limit is per...
Read more >
Create a Local Ubuntu Repository using Apt-Mirror and ...
Ubuntu repository available within your LAN, saving you future bandwidth and avoiding redundant downloads.
Read more >
Local Mirror - Gentoo Wiki
Using public mirrors is simply a waste of bandwidth at both ends. Syncing only one machine against a public mirror and all others...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found