After updating `psycopg2`, installing meltano now requires `pg_config` to be present on system

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/2063

Originally created by @DouweM on 2020-05-26 20:49:52

---

Technically introduced by https://gitlab.com/meltano/meltano/-/merge_requests/1665

Failure in job that didn’t have pg_config: https://gitlab.com/meltano/meltano/-/jobs/568513947

Error message

    digitalocean:   Downloading psycopg2-2.8.5.tar.gz (380 kB)
==> digitalocean:     ERROR: Command errored out with exit status 1:
==> digitalocean:      command: /var/meltano/.venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3k2ro1j7/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3k2ro1j7/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-nlhb7bn4
==> digitalocean:          cwd: /tmp/pip-install-3k2ro1j7/psycopg2/
==> digitalocean:     Complete output (23 lines):
==> digitalocean:     running egg_info
==> digitalocean:     creating /tmp/pip-pip-egg-info-nlhb7bn4/psycopg2.egg-info
==> digitalocean:     writing /tmp/pip-pip-egg-info-nlhb7bn4/psycopg2.egg-info/PKG-INFO
==> digitalocean:     writing dependency_links to /tmp/pip-pip-egg-info-nlhb7bn4/psycopg2.egg-info/dependency_links.txt
==> digitalocean:     writing top-level names to /tmp/pip-pip-egg-info-nlhb7bn4/psycopg2.egg-info/top_level.txt
==> digitalocean:     writing manifest file '/tmp/pip-pip-egg-info-nlhb7bn4/psycopg2.egg-info/SOURCES.txt'
==> digitalocean:
==> digitalocean:     Error: pg_config executable not found.
==> digitalocean:
==> digitalocean:     pg_config is required to build psycopg2 from source.  Please add the directory
==> digitalocean:     containing pg_config to the $PATH or specify the full executable path with the
==> digitalocean:     option:
==> digitalocean:
==> digitalocean:         python setup.py build_ext --pg-config /path/to/pg_config build ...
==> digitalocean:
==> digitalocean:     or with the pg_config option in 'setup.cfg'.
==> digitalocean:
==> digitalocean:     If you prefer to avoid building psycopg2 from source, please install the PyPI
==> digitalocean:     'psycopg2-binary' package instead.
==> digitalocean:
==> digitalocean:     For further information please check the 'doc/src/install.rst' file (also at
==> digitalocean:     <https://www.psycopg.org/docs/install.html>).
==> digitalocean:
==> digitalocean:     ----------------------------------------
==> digitalocean: ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Our users could run into this as well, which is not ideal because we want to make installing Meltano as easy as possible.

[visch]

@DouweM Hi, we are currently using the image version meltano/meltano:latest-python3.9, and we are stumbling upon the same error message as shown above. How do we resolve this?

@paulorijnberg Here’s my working docker file with the changes I had to add for this, hope it helps!

RUN apt install libpq-dev -y is the difference

# registry.gitlab.com/meltano/meltano:latest is also available in GitLab Registry
ARG MELTANO_IMAGE=meltano/meltano:latest
FROM $MELTANO_IMAGE

WORKDIR /project
RUN apt-get update
RUN apt install libpq-dev -y

# Install any additional requirements
COPY ./requirements.txt .
RUN pip install -r requirements.txt

# Install all plugins into the `.meltano` directory
COPY ./meltano.yml .
RUN meltano install

# Pin `discovery.yml` manifest by copying cached version to project root
RUN cp -n .meltano/cache/discovery.yml . 2>/dev/null || :

# Don't allow changes to containerized project files
ENV MELTANO_PROJECT_READONLY 1

# Copy over remaining project files
COPY . .

# Expose default port used by `meltano ui`
EXPOSE 5000

ENTRYPOINT ["meltano"]

[paulorijnberg]

@DouweM Hi, we are currently using the image version meltano/meltano:latest-python3.9, and we are stumbling upon the same error message as shown above. How do we resolve this?