PKG_CONFIG_PATH not exported from windows to msys2

This is a new error that I’m getting in GitHub CI. It started about 2 weeks ago. Seems like cygpath might be provided by git for windows, which I have never explicitly installed… Maybe GitHub changed something about their environment and cygpath is no longer in there?

/usr/bin/bash: line 1: cygpath: command not found
/usr/bin/bash: line 1: source: filename argument required
source: usage: source filename [arguments]
Error: Process completed with exit code 1.

[eyal0]

@eine Okay, I found a solution to my problem. I’ll describe it here and the different things that I tried. Hopefully it will save someone time in the future. If you don’t have time to read it, here’s the trick: Use ~/.bash_profile instead of $GITHUB_ENV for everything.

Problem

Attempt 1: $GITHUB_ENV

We want to use a single GitHub CI for all builds, macos, Windows, and Ubuntu. This makes our CI smaller and more manageable. macos and unix are pretty easy because they are mostly compatible but Windows is a little trickier, especially about the environment variables! The issue is that if you put something like this in your CI:

- name: Set env variable FOO to bar
  run: |
      echo "FOO=bar" >> $GITHUB_ENV

It will set the variable in the environments for macos and for ubuntu, but it won’t work the same for Windows. That’s because it is setting the environment variable in the windows environment, which is, like the powershell. But you want the environment variable set inside of msys2.

Attempt 2: path-type: inherit

The solution should be to use path-type: inherit. You can read about how it works. If you look at the code for it here and here, you’ll see that inherit only affects PATH and not other things that you might set, like PKG_CONFIG_PATH. That’s a bummer because we’d like those to work, too.

Attempt 3: source a different file

One solution would be to instead write to a different file, maybe called ~/my_env and then source it at each command. So you would need to modify every single command in your CI file to have a source line at the top. Like this:

       - run: |
         source ~/.my_env
         do_the_rest.sh

That’s annoying! We don’t want to have to do that.

Attempt 4: source it in the shell

GitHub will let you change the shell command for your platform. So you might try something like this:

    defaults:
      run:
        shell: source ~/my_env; /bin/bash {0}

Now it should source your file before running the shell each time. I tried that but it didn’t work. GitHub was complaining that source wasn’t an executable file on the disk. That’s true, source is a command that bash know about, it’s not a file that can be executed.

Attempt 5: Put it in .bashrc

We know that bash will always run .bashrc before it starts. So we could just put the lines that we want in .bashrc. However, one problem: .bashrc is only run for interactive shells. This is described in the man page for bash:

       When an interactive shell that is not a login shell is started, bash reads and executes commands from /etc/bash.bashrc and ~/.bashrc, if these files exist.  This may be inhibited by  us‐
       ing the --norc option.  The --rcfile file option will force bash to read and execute commands from file instead of /etc/bash.bashrc and ~/.bashrc.

So we need to make bash interactive. So we need to modify the shell to be interactive.

    defaults:
      run:
        shell: source ~/my_env; /bin/bash -i {0}

This causes other problems, though, because if the shell is interactive then some of the stuff that you do in your CI will behave differently and your tests will fail in a weird way. So this is no good.

Attempt 6: Put it in .bash_login.

~/.bash_login is run only for login shells. And we can fake that by running bash with a -l. So that’s a good technique for us!

Solution

So in the end, here’s what the CI script could looks like:

jobs:
  my_job:
    name: ${{ matrix.os }}_job
    strategy:
      matrix:
        os: [ubuntu, macos, windows]
        include:
          - os: windows
            shell: msys2 {0}
          - os: ubuntu
            shell: '/usr/bin/bash -l -e -o pipefail {0}'
          - os: macos
            shell: '/bin/bash -l -e -o pipefail {0}'
    runs-on: ${{ matrix.os }}-latest
    defaults:
      run:
        shell: ${{ matrix.shell }}
    steps:
    - name: Setup paths and env
      run: |
        rm -f ~/.bash_profile
        echo "export FOO=bar" >> ~/.bash_profile

What this does is:

• Create a matrix of jobs, where the variable os is one of ubuntu, windows, and macos. So matrix, if we converted it to json, would be 3 jobs, each with a different value of os. Here’s what the job matrix looks like in yaml:

          - os: windows
          - os: ubuntu
          - os: macos

• Now we have the include lines, which attach the shell variable to all elements of the matrix with os that matches. This way, if you have an even more complex matrix with lots of rows you don’t have to attach the shell to each one. This is what we have now in yaml for our matrix.

          - os: windows
            shell: msys2 {0}
          - os: ubuntu
            shell: '/usr/bin/bash -l -e -o pipefail {0}'
          - os: macos
            shell: '/bin/bash -l -e -o pipefail {0}'

• Then we set runs on an the default shell. Those come from the matrix so we’ll have a job running windows with the windows shell, a job running ubuntu with the ubuntu shell, etc.
• Our first step is to delete the ~/.bash_profile. It starts as all comments except for a line that causes it to exit when we are not interactive. Any lines that we place after that won’t be executed because we are not interactive, so we need to remove that. Might as well just remove it all, there is nothing useful in there. (If you want to check for yourself, put cat .bash_profile into a CI command and see it in the logs.)
• Now we are ready to add environment variables. We append them to the ~/.bash_profile. I used export but I’m not totally sure if that is necessary. No harm in it anyway.

That’s it! Now a single CI will work for windows and ubuntu and macos. The pcb2gcode project is using this successfully with a fairly complicated build for all three platforms and they all use the same steps, more or less.

[eyal0]

You can see here where PATH gets special handling: https://github.com/msys2/MSYS2-packages/blob/915946a637e1f2b7e26e32782f3af322009293db/filesystem/profile#L28-L45

PKG_CONFIG_PATH does not: https://github.com/msys2/MSYS2-packages/blob/915946a637e1f2b7e26e32782f3af322009293db/filesystem/profile#L53