CLI plugin system does not work on NixOS

Much of the functionality of the CLI relies on a plugin system, esp. the hasura metadata ... commands. The plugin system AFAICT is built around downloading binaries for each plugin which are maintained in ~/.hasura. Unfortunately these binaries are dynamically linked and do not work across all Linux distros, in particular they don’t work on NixOS where /lib64/ld-linux-x86-64.so.2 is not generally present.

Here’s an example of what I mean:

[skainswo@ip-172-31-5-175:~]$ nix-shell -p hasura-cli file

[nix-shell:~]$ hasura plugins install cli-ext
INFO "can't install, plugin is already installed"  name=cli-ext

[nix-shell:~]$ cd .hasura/plugins/store/cli-ext/v1.4.0-alpha.2/

[nix-shell:~/.hasura/plugins/store/cli-ext/v1.4.0-alpha.2]$ ./cli-ext-hasura-linux
bash: ./cli-ext-hasura-linux: No such file or directory

[nix-shell:~/.hasura/plugins/store/cli-ext/v1.4.0-alpha.2]$ file cli-ext-hasura-linux
cli-ext-hasura-linux: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, BuildID[sha1]=2106a959eb86690fe4628f369dd4b325b515ed02, with debug_info, not stripped

[nix-shell:~/.hasura/plugins/store/cli-ext/v1.4.0-alpha.2]$ ldd cli-ext-hasura-linux
	linux-vdso.so.1 (0x00007ffe722c2000)
	libdl.so.2 => /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libdl.so.2 (0x00007f88b1e05000)
	librt.so.1 => /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/librt.so.1 (0x00007f88b1dfb000)
	libstdc++.so.6 => not found
	libm.so.6 => /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libm.so.6 (0x00007f88b1cba000)
	libgcc_s.so.1 => /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libgcc_s.so.1 (0x00007f88b1ca0000)
	libpthread.so.0 => /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libpthread.so.0 (0x00007f88b1c7f000)
	libc.so.6 => /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6 (0x00007f88b1abe000)
	/lib64/ld-linux-x86-64.so.2 => /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib64/ld-linux-x86-64.so.2 (0x00007f88b1e0c000)

Note that libstdc++.so.6 is not resolved. However, I’ve tested that even with libstdc++.so.6 in LD_LIBRARY_PATH, the binary does not run, likely because of the /lib64/ld-linux-x86-64.so.2 interpreter issue.

This is especially problematic when trying to run eg

[nix-shell:~/dev/cuddlefish/hasura]$ hasura metadata apply
FATA[0001] failed to apply metadata: cannot apply metadata on the database: cannot build actions from metadata: cannot fetch plugin manfiest cli-ext: plugin version is not available

which makes the CLI mostly useless for me.

I’m guessing the simplest solution would be to simply ship statically linked executables for plugins. This should reduce the complexity of running on different systems, and (I think) get the plugin system working on NixOS. Alternatively, the metadata functionality could be moved into the main CLI binary itself, which AFAIU would seem to make more sense.

Related:

• https://github.com/hasura/graphql-engine/issues/5449
• https://github.com/hasura/graphql-engine/discussions/6573
• https://github.com/NixOS/nixpkgs/issues/113756

[silky]

this issue should definitely not be closed; it’s still an issue 🥲 🥲 🥲 🥲

– edit: if any poor soul is reading this and desparate i’ll note that @datakurre from over here - https://github.com/NixOS/nixpkgs/issues/113756#issuecomment-820214578 - provided a way to build the required executable on nixos that works fine; but is quite complex.

what i ended up doing was just ( prepare yourself for sadness ) use a docker image and run the executable there. it works, but it’s sad. a road i tried that lead to more sadness was to build the cli-ext project from the codebase directly (under nix) but this lead to errors, so i gave up fairly immediately.

– edit: double update; i’ll report that the docker exetuable approach isn’t perfect, because, you can’t run hasura console with it, because hasura only has ONE url for accessing the endpoints, and if it’s running on docker, you’ll probably want two (for various reasons that you can work out yourself; i.e. your hasura instance will likely be in your docker network, etc, etc, etc )

[samuela]

I can confirm that this is an issue with NixOS. Just tried the exact same thing on macOS and it ran just fine.