[Proposal] Middleware hosting and CI/CD automation

Hi, I’ve just briefly talked with @yusukebe about the way to host 3rd party middleware under honojs GitHub organization and publish them to npm and deno registries. I recently went through Jenkins plugin development processes and was really impressed how they manage such a huge ecosystem by using the similar concept. I believe we can get some insights from them and build a smart solution. Here is my rough proposal. Thank you for reading this proposal in advance!

Summary

We can create a dedicated GitHub and npmjs team for each 3rd party middleware and manage fine-grained access control for the hosted repository and the published package. We can start from whole manual workflow and gradually automate them by implementing several bots.

Source code hosting

We’d like to host all middleware as a GitHub repository under honojs GitHub organization to give more credibility to our users comparing with the ones hosted under arbitrary user/organization. The issue here is how to manage membership because the authors of 3rd party middleware shouldn’t have write access to other honojs repository. Jenkins (jenkinsci organization) hosts all 3rd party plugins by following the procedure below:

1. A contribute creates a public GitHub repository for the plugin under any GitHub user/organization.
2. Once they finish the initial implementation, they cut an GitHub issue for “Hosting request” like https://github.com/jenkins-infra/repository-permissions-updater/issues/2708
3. Some automated processes evaluate the request. Also, a few community members review it, then approve/reject.
4. Then, the automated process generally does:
   1. Fork the original repository under jenkinsci organization. Ask the requester to delete their original repository.
   2. Creates a dedicated GitHub team like this and add the requested author(s). The list is managed by their infra repository like this
   3. Give Admin permission of the repository to the new GitHub team

After these steps, the author(s) have the full permission of the hosted repository like they had before beautifully. I believe we can mimic the way Jenkins does, manually first and automatically once we implement the bots.

Package publishing

We’d like to publish all middleware under @honojs scope for the same reason as the source code hosting. The issue here is how to give fine-grained permission to the authors of 3rd party middleware. Let’s look into the Jenkins example again:

1. In the “Hosting request”, they also give release permissions to the requested members which allows them to manipulate Maven artifacts hosted by Jenkins team. This is done by giving the package release permission on the hosted repository website via bot.
2. They also allow the authors to fully automate the release process by giving some credentials to the GitHub repository via GitHub Actions Secrets.

As you can see above, the concept is similar to the source code hosting and we can apply this idea to our package publishing as well. npmjs supports teams under organizations and we can assign selected packages and members to the team like GitHub’s team. To automate the publishing, we should create a machine user per repository on npmjs, invite it to the npmjs’s team and generate an automation token (allowed to bypass 2FA), and give the token to the repository via GitHub Actions Secrets.

Note: We also want to do the similar thing for Deno but https://deno.land/x doesn’t support scope or namespace as far as I know. So, we might need to consider to use other hosting like https://esm.sh/. We don’t discuss Deno now in this issue but will do later.

Other opinions

This is my preference but I would recommend you to name all middleware repositories prefixed or suffixed by middleware or something e.g. honojs/middleware-sentry or honojs/sentry-middleware. This enables us to identify which repositories are for middleware very easily. Jenkins requests all new plugins to name suffixed by -plugin. This also can avoid unnecessary future name conflicts.

References

Original discussion: https://github.com/honojs/hono/issues/350

---

Feel free to comment any feedback! I’m really interested in the implementation of these processes if you like.

[yusukebe]

Hi @riywo !

My point here is that there is no concept like scope of npmjs on deno.land/x. So, we can’t secure a package name for a new middleware if someone already created the same package, just like the global scope of npmjs or majority of package registries like CPAN, rubygems, etc.

You are right!! I reconsidered, and also, I think that the “deno.land/x” has a problem it does not have a “scope,”. And we can’t get the module name if it is already taken.

Now, we have good news. The recently released Deno v1.25 includes an experimental feature that allows npm modules to be imported without using a CDN like “https://esm.sh/”

• Deno 1.25 Release Notes https://deno.com/blog/v1.25

import { Hono } from 'npm:hono'
import { poweredBy } from 'npm:hono/powered-by'

const app = new Hono()
app.use('*', poweredBy())

app.get('/', (c) => c.text('Hello Deno!'))

Deno.serve(app.fetch)

This code will really work!!

Of course, this is an experimental feature, and it will be yet to be officially introduced. Although that, I think we would like to move to this approach in the future. Until then, using a CDN like “https://esm.sh/” is a good idea.

In addition, if we use the CDN or the new feature of Deno, we do not need to do “denoify”. It is used to transform for Deno code. This process was a little bit tricky.

So you are right!!

I would like to discuss this continue in another Issue.

In conclusion, I think we do not need to consider how to publish on “deno.land/x” right now. First, we should make the way to publish to npmjs.

[riywo]

@yusukebe @Code-Hex Thank you so much for your feedback!

Can we create a machine user with npmjs? When I looked for a way to do it, I couldn’t find it.

People just create a new user with a shared email address and password like this https://www.npmjs.com/~autifyhq-release

I’d like to add something about Deno. There is not much to do when we publish a module in deno.land/x. To register, enter the module name and set up a webhook in the GitHub repository, referring to the following.

My point here is that there is no concept like scope of npmjs on deno.land/x. So, we can’t secure a package name for a new middleware if someone already created the same package, just like the global scope of npmjs or majority of package registries like CPAN, rubygems, etc.