Postgres 9 service pg_ctl command is returning invalid path

[sinemetu1]

Hello,

Our travis builds are using the postgresql service feature. For some reason builds in the past few days have been failing because pg_ctl --bindir reports an invalid path (for version 10? 🤔). I've created a travis build and yaml to reproduce this. Here is the .travis.yaml:

sudo: false
language: python
python: 3.6

services:
    - postgresql

script:
    - which pg_ctl
    - echo $PATH
    - pg_config --bindir
    - ls -lh /usr/lib/postgresql/
    - ls -lh /usr/lib/postgresql/10/bin/
    - ls -lh /usr/lib/postgresql/10/bin/pg_ctl

Here's the travis build: https://travis-ci.org/sinemetu1/test-pg-travis/builds/325130012

And the related section of the logs:

...
$ which pg_ctl
The command "which pg_ctl" exited with 1.
$ echo $PATH
/home/travis/virtualenv/python3.6.3/bin:/home/travis/bin:/home/travis/.local/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.9.1/bin:/home/travis/.kiex/elixirs/elixir-1.4.5/bin:/home/travis/.kiex/bin:/home/travis/.rvm/gems/ruby-2.4.1/bin:/home/travis/.rvm/gems/ruby-2.4.1@global/bin:/home/travis/.rvm/rubies/ruby-2.4.1/bin:/home/travis/gopath/bin:/usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/home/travis/.gimme/versions/go1.7.4.linux.amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin
The command "echo $PATH" exited with 0.
$ pg_config --bindir
/usr/lib/postgresql/10/bin
The command "pg_config --bindir" exited with 0.
$ ls -lh /usr/lib/postgresql/
total 0
drwxr-xr-x 4 root root 26 Dec  5 19:59 9.2
drwxr-xr-x 4 root root 26 Dec  5 19:59 9.3
drwxr-xr-x 4 root root 26 Dec  5 20:00 9.4
drwxr-xr-x 4 root root 26 Dec  5 20:00 9.5
drwxr-xr-x 4 root root 26 Dec  5 20:00 9.6
The command "ls -lh /usr/lib/postgresql/" exited with 0.
$ ls -lh /usr/lib/postgresql/10/bin/
ls: cannot access /usr/lib/postgresql/10/bin/: No such file or directory
...

Thanks for your time and help!

[BanzaiMan]

This feels like a packaging issue. We are installing postgresql-common, which provides this file. Even when there is no PostgreSQL 10 package installed on the system, the binary /usr/bin/pg_config.libpq-dev has strings pointing to version 10.

travis@42efb11a6c57:~$ dpkg -l | grep postgres
ii  pgdg-keyring                             2017.3                                     all          keyring for apt.postgresql.org
ii  postgresql-9.2                           9.2.24-1.pgdg14.04+1                       amd64        object-relational SQL database, version 9.2 server
ii  postgresql-9.2-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.2 -- scripts
ii  postgresql-9.3                           9.3.20-1.pgdg14.04+1                       amd64        object-relational SQL database, version 9.3 server
ii  postgresql-9.3-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.3 -- scripts
ii  postgresql-9.4                           9.4.15-1.pgdg14.04+1                       amd64        object-relational SQL database, version 9.4 server
ii  postgresql-9.4-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.4 -- scripts
ii  postgresql-9.5                           9.5.10-1.pgdg14.04+1                       amd64        object-relational SQL database, version 9.5 server
ii  postgresql-9.5-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.5 -- scripts
ii  postgresql-9.6                           9.6.6-1.pgdg14.04+1                        amd64        object-relational SQL database, version 9.6 server
ii  postgresql-9.6-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.6 -- scripts
ii  postgresql-client                        9.3+154ubuntu1.1                           all          front-end programs for PostgreSQL (supported version)
ii  postgresql-client-9.2                    9.2.24-1.pgdg14.04+1                       amd64        front-end programs for PostgreSQL 9.2
ii  postgresql-client-9.3                    9.3.20-0ubuntu0.14.04                      amd64        front-end programs for PostgreSQL 9.3
ii  postgresql-client-9.4                    9.4.15-1.pgdg14.04+1                       amd64        front-end programs for PostgreSQL 9.4
ii  postgresql-client-9.5                    9.5.10-1.pgdg14.04+1                       amd64        front-end programs for PostgreSQL 9.5
ii  postgresql-client-9.6                    9.6.6-1.pgdg14.04+1                        amd64        front-end programs for PostgreSQL 9.6
ii  postgresql-client-common                 188.pgdg14.04+1                            all          manager for multiple PostgreSQL client versions
ii  postgresql-common                        188.pgdg14.04+1                            all          PostgreSQL database-cluster manager
ii  postgresql-contrib-9.2                   9.2.24-1.pgdg14.04+1                       amd64        additional facilities for PostgreSQL
ii  postgresql-contrib-9.3                   9.3.20-1.pgdg14.04+1                       amd64        additional facilities for PostgreSQL
ii  postgresql-contrib-9.4                   9.4.15-1.pgdg14.04+1                       amd64        additional facilities for PostgreSQL
ii  postgresql-contrib-9.5                   9.5.10-1.pgdg14.04+1                       amd64        additional facilities for PostgreSQL
ii  postgresql-contrib-9.6                   9.6.6-1.pgdg14.04+1                        amd64        additional facilities for PostgreSQL
travis@42efb11a6c57:~$ strings /usr/bin/pg_config.libpq-dev | grep postgres
/usr/lib/postgresql/10/bin
Report bugs to <pgsql-bugs@postgresql.org>.
'--with-icu' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' '--with-tclconfig=/usr/lib/x86_64-linux-gnu/tcl8.6' '--with-includes=/usr/include/tcl8.6' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/10/man' '--docdir=/usr/share/doc/postgresql-doc-10' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/10' '--bindir=/usr/lib/postgresql/10/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' 'CFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' '--with-selinux' 'CPPFLAGS=-D_FORTIFY_SOURCE=2'
postgres-10
/usr/share/postgresql/10
/etc/postgresql-common
/usr/include/postgresql
/usr/lib/postgresql/10/lib
/usr/share/postgresql/10/man
/usr/include/postgresql/10/server
/usr/share/doc/postgresql-doc-10

[BanzaiMan]

libpq-dev points to version 10:

travis@42efb11a6c57:~$ dpkg -l | grep libpq
ii  libpq-dev                                10.1-1.pgdg14.04+1                         amd64        header files for libpq5 (PostgreSQL library)
ii  libpq5:amd64                             10.1-1.pgdg14.04+1                         amd64        PostgreSQL C client library

[BanzaiMan]

Looking at the content of /usr/bin/pg_config (which is just a wrapper), I believe that it is (somewhat) assuming that postgresql-server-dev-X.Y is also installed.

travis@988f659570cf:~$ sudo apt-get install postgresql-server-dev-9.4
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  postgresql-server-dev-9.4
0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded.
Need to get 645 kB of archives.
After this operation, 3,344 kB of additional disk space will be used.
Get:1 http://apt.postgresql.org/pub/repos/apt trusty-pgdg/main amd64 postgresql-server-dev-9.4 amd64 9.4.15-1.pgdg14.04+1 [645 kB]
Fetched 645 kB in 0s (685 kB/s)
Selecting previously unselected package postgresql-server-dev-9.4.
(Reading database ... 72205 files and directories currently installed.)
Preparing to unpack .../postgresql-server-dev-9.4_9.4.15-1.pgdg14.04+1_amd64.deb ...
Unpacking postgresql-server-dev-9.4 (9.4.15-1.pgdg14.04+1) ...
Setting up postgresql-server-dev-9.4 (9.4.15-1.pgdg14.04+1) ...
travis@988f659570cf:~$ pg_config --bindir
/usr/lib/postgresql/9.4/bin

Unfortunately, we can't install these packages for all the versions we install on our build images, because /usr/bin/pg_config always runs the latest version. (See https://stackoverflow.com/q/30143046/136333.)

Ultimately, I think this is a bug with PostgreSQL.

[sinemetu1]

Ah okay. Thanks for the help!

[BanzaiMan]

If you are interested in getting useful output from pg_config, I think you can install postgresql-server-dev-9.4 (or whatever version you are using); e.g.,

addons:
  apt:
    packages:
      - postgresql-server-dev-9.4

[sinemetu1]

Using the addon worked for us. Thanks!