install 9.15.1 fail

Error: ENOENT: no such file or directory, chmod '.../node_modules/_highlight.js@[e-mail]/tools/build.js'

missing tools/build.js file.

[marcoscaceres]

incident report

What happened?

About a week ago we merged #1951, which made changes to the tools/build.js utility by excluding certain files (docs, demos, and other things). The PR’s intention was to have a way to build a highlight bundle from the command line, but to not build the docs and demos that are not needed on CI environments.

Although we’d been thoroughly testing this binary in production in another project for over a year, we’d been installing highlight.js over a github: URL. Unknown to me, the highlight.js npm publish script only included certain files: The script is on an ex-maintainer’s private server, so I’d incorrectly assumed that it was publishing the repository as is. This assumption was wrong and led to things breaking.

When 9.15.1 release went live, it was missing tools/build.js and possibly other things causing depedents to break.

What we did in response

At first, I thought the problem was that the bin path was maybe wrong. However, thanks to helpful https://github.com/highlightjs/highlight.js/issues/1984#issuecomment-466938694 from @netanelgilad, it was pointed out that the problem was with how the package was being published: it was missing directories/files.

As a result, I reverted #1951. However, this was insufficient because we’d changed our release script to work with the new tools/build.js. We then needed to revert some changes to the release script.

After a few bad releases, we then successfully published 9.15.5.

What we learned

Keeping in mind that the maintainers are new to the project:

• the published package does not reflect what’s in the repository.

What could we do to prevent this from happening again

We need to figure out a way to pre-publish (or publish as “beta”) a package, and then try to install the package in a mock project to make sure things still work as expected.

If folks have suggestion for how to do that - or what a good setup might be to test what’s actually published before it goes live, I’d be open to suggestions.

[isagalaev]

I would like if you revert the change. It is currently blocking a lot of build from other people

On a personal note, let me take this as an opportunity to explain something about the current sorry state of relationship between businesses and open source projects. (Yeah, I know, but people still don’t get it.)

highlight.js is not a business, it’s a hobby.

It means that whatever gets pushed to this repository or npm should be assumed to be the result of someone having fooled around and gone away for a weekend with their family. Or for a busy working day at their job.

If a business has made a decision to rely on this artifact for anything requiring any sort of stability (i.e. “blocking a lot of build from other people”), it made a stupid and uninformed decision. Or more realistically, it simply relies on maintainers feeling ashamed enough to quickly fix problems when they happen. Even more realistically, it just accept the fact that their engineers are going to deal with maintainers by soliciting free support, because it has always worked this way. I, for one, don’t feel any urge at all supporting someone’s misplaced expectations 😃

So, dear fellow engineers, please take this build hiccup as an opportunity to explain your particular business people that their entire intellectual property is a thin layer on top of a shaky foundation of open-source code lazily maintained by hobbyists or paid for by other businesses having their own goals in mind. Mention the leftpad story for more effect. So if they really want stability they have to invest in it. By, for example, hiring engineers to deal with myriad of dependencies, maintain local stable forks, contribute patches upstream, or whatever — the key point is that it should not look like it “just works” on fairy dust.