Deploy PHP7 Deb/RPM packages #117

cyanide-burnout · 2017-04-16T11:36:14Z

Please build the package and place it into tarantool package repository

Totktonada · 2018-12-12T21:27:18Z

Plan:

add actual distros with php >= 7.1 (debian, ubuntu, centos, fedora);
add builds for different php versions under different package names (-php71 suffix or like so—need to see another php extensions).

Verify `packpack` based builds for CentOS and Fedora distributions using Travis CI. Deployment is not enabled here: it is subject for further commits. The key point is that we'll build the extension for php version that is provided by an OS itself. Why don't build against PHP versions from widely used Remi's repository? There are two reasons: * It will introduce dependency on a third-party repository for our third-party repository, at least in context of php-tarantool RPM packages. I guess it would quite unobvious from a user perspective. * We don't control when PHP ABI version will be changed in Remi's repositories. Is there a politic about this? I don't aware of it. We can get non-working packages after an update of PHP version in Remi's repo. So the decision for now is to provide packages for a PHP version provided by a distribution itself (from its base repositories). I plan to contact Remi and consult how to handle everything in a way that would be convenient for a user. Maybe it would be good to provide sources, spec and build tools from our side and ask Remi to include the extension into his repositories. Part of #117

Provided the script to build a RPM / Deb package, install it inside a docker container based on an appropriate image and verify that the main class of the extension (Tarantool) is available from the PHP interpreter. The script builds a package just like it is performed in CI: using packpack. It should be available in PATH directories. Only RPM packages are here now, but the following commits will enable Deb packages too. Part of #117

I guess it is the behaviour, which is expected by a user: when an extension is installed it should be available for use. Part of #117

Verify Deb packages on distributions, which provides PHP 7.0: it is Debian Stretch and Ubuntu Xenial. Now the Debian build files (debian/ directory) allow to build against PHP 7.0 only (not PHP 7.1+), because of hardcoded package name and installation paths. That is why newer Debian and Ubuntu distributions are not enabled here. It will be fixes in the following commits. Part of #117

Preprocess debian/control file to set proper package name like php7.3-tarantool. Use the package name in debian/rules file to determine installation paths. Removed php7.0-dev dependency, because php-all-dev already depends on proper php7.?-dev package. This change allows to build a Deb package for distributions, which provides PHP version other then 7.0. Those builds will be enabled in CI in further commits. Details ------- It seems it is not possible to build a package with a conditional name (depending of a distribution). `man 5 deb-substvars` explicitly states that 'Package' field in debian/control can be parametrized using a substitution varible. Another possible way would be define several 'Package' fields in debian/control and build only one for each distribution. However I don't find a way to build only one package using `debuild` (it is used inside `packpack`). So it seems that the preprocessing is the only way to build, say, php7.0-tarantool package for Debian Stretch and php7.3-tarantool for Debian Buster. The preprocessing is performed from debian/prebuild.sh script, which is invoked by `packpack`. So nothing changed in the build process from a caller perspective. Part of #117

The distributions which provides PHP 7.1+ were blocked, because 7.0 (the PHP version) was hardcoded in package name and installation paths. Now the problem is resolved and we can verify those packages in CI (and in the local verification script test.pkg.all.sh). Part of #117

Verify `packpack` based builds for CentOS and Fedora distributions using Travis CI. Deployment is not enabled here: it is subject for further commits. The key point is that we'll build the extension for php version that is provided by an OS itself. Why don't build against PHP versions from widely used Remi's repository? There are two reasons: * It will introduce dependency on a third-party repository for our third-party repository, at least in context of php-tarantool RPM packages. I guess it would quite unobvious from a user perspective. * We don't control when PHP ABI version will be changed in Remi's repositories. Is there a politic about this? I don't aware of it. We can get non-working packages after an update of PHP version in Remi's repo. So the decision for now is to provide packages for a PHP version provided by a distribution itself (from its base repositories). I plan to contact Remi and consult how to handle everything in a way that would be convenient for a user. Maybe it would be good to provide sources, spec and build tools from our side and ask Remi to include the extension into his repositories. Part of #117

Provided the script to build a RPM / Deb package, install it inside a docker container based on an appropriate image and verify that the main class of the extension (Tarantool) is available from the PHP interpreter. The script builds a package just like it is performed in CI: using packpack. It should be available in PATH directories. Only RPM packages are here now, but the following commits will enable Deb packages too. Part of #117

I guess it is the behaviour, which is expected by a user: when an extension is installed it should be available for use. Part of #117

Verify Deb packages on distributions, which provides PHP 7.0: it is Debian Stretch and Ubuntu Xenial. Now the Debian build files (debian/ directory) allow to build against PHP 7.0 only (not PHP 7.1+), because of hardcoded package name and installation paths. That is why newer Debian and Ubuntu distributions are not enabled here. It will be fixes in the following commits. Part of #117

Preprocess debian/control file to set proper package name like php7.3-tarantool. Use the package name in debian/rules file to determine installation paths. Removed php7.0-dev dependency, because php-all-dev already depends on proper php7.?-dev package. This change allows to build a Deb package for distributions, which provides PHP version other then 7.0. Those builds will be enabled in CI in further commits. Details ------- It seems it is not possible to build a package with a conditional name (depending of a distribution). `man 5 deb-substvars` explicitly states that 'Package' field in debian/control can be parametrized using a substitution varible. Another possible way would be define several 'Package' fields in debian/control and build only one for each distribution. However I don't find a way to build only one package using `debuild` (it is used inside `packpack`). So it seems that the preprocessing is the only way to build, say, php7.0-tarantool package for Debian Stretch and php7.3-tarantool for Debian Buster. The preprocessing is performed from debian/prebuild.sh script, which is invoked by `packpack`. So nothing changed in the build process from a caller perspective. Part of #117

The distributions which provides PHP 7.1+ were blocked, because 7.0 (the PHP version) was hardcoded in package name and installation paths. Now the problem is resolved and we can verify those packages in CI (and in the local verification script test.pkg.all.sh). Part of #117

Totktonada · 2020-06-02T23:26:56Z

NB: For ones who needs custom builds for php-7.3 or php-7.4 from ppa:ondrej/php repository for Ubuntu Bionic.

Apply ones of the following patches on top of php7-v2 branch:

php-7.3:

diff --git a/debian/prebuild.sh b/debian/prebuild.sh
index c17d14c..b33595f 100755
--- a/debian/prebuild.sh
+++ b/debian/prebuild.sh
@@ -11,6 +11,12 @@ SUDO="sudo -E"
 ${SUDO} apt-get update > /dev/null
 ${SUDO} apt-get -y install php-all-dev
 
+# Build against php7.3 on Ubuntu Bionic.
+${SUDO} apt-get install software-properties-common
+${SUDO} add-apt-repository -y ppa:ondrej/php
+${SUDO} apt-get update > /dev/null
+${SUDO} apt-get install -y php7.3-dev
+
 phpversion=$(php-config --version | sed 's/^\([0-9]\+\.[0-9]\).*/\1/')
 
 cd /build/php-tarantool-*
diff --git a/test.pkg.all.sh b/test.pkg.all.sh
index fd74685..388cc62 100755
--- a/test.pkg.all.sh
+++ b/test.pkg.all.sh
@@ -3,27 +3,9 @@
 set -exu  # Strict shell (w/o -o pipefail)
 
 distros="
-    el:8
-    fedora:25
-    fedora:26
-    fedora:27
-    fedora:28
-    fedora:29
-    fedora:30
-    fedora:31
-    debian:stretch
-    debian:buster
-    ubuntu:xenial
     ubuntu:bionic
-    ubuntu:disco
-    ubuntu:eoan
 "
 
-if ! type packpack; then
-    echo "Unable to find packpack"
-    exit 1
-fi
-
 for distro in $distros; do
     export OS="${distro%%:*}"
     export DIST="${distro#*:}"
@@ -31,9 +13,6 @@ for distro in $distros; do
         export OS=centos
     fi
 
-    rm -rf build
-    packpack
-
     docker run \
         --volume "$(realpath .):/tarantool-php" \
         --workdir /tarantool-php                \
diff --git a/test.pkg.sh b/test.pkg.sh
index 3c1c271..e80d587 100755
--- a/test.pkg.sh
+++ b/test.pkg.sh
@@ -29,8 +29,14 @@ fi
 # Update available packages list.
 ${PM_SYNC}
 
+# Verify against php7.3 on Ubuntu Bionic.
+apt-get -y install software-properties-common
+add-apt-repository -y ppa:ondrej/php
+apt-get update > /dev/null
+
 # Install php interpreter.
-${PM_INSTALL_NAME} php
+apt-get install -y php7.3
+php --version
 
 # Install php-tarantool package.
 ${PM_INSTALL_FILE} build/${PKG_GLOB}

php-7.4:

diff --git a/debian/prebuild.sh b/debian/prebuild.sh
index c17d14c..b33595f 100755
--- a/debian/prebuild.sh
+++ b/debian/prebuild.sh
@@ -11,6 +11,12 @@ SUDO="sudo -E"
 ${SUDO} apt-get update > /dev/null
 ${SUDO} apt-get -y install php-all-dev
 
+# Build against php7.4 on Ubuntu Bionic.
+${SUDO} apt-get install software-properties-common
+${SUDO} add-apt-repository -y ppa:ondrej/php
+${SUDO} apt-get update > /dev/null
+${SUDO} apt-get install -y php7.4-dev
+
 phpversion=$(php-config --version | sed 's/^\([0-9]\+\.[0-9]\).*/\1/')
 
 cd /build/php-tarantool-*
diff --git a/test.pkg.all.sh b/test.pkg.all.sh
index fd74685..388cc62 100755
--- a/test.pkg.all.sh
+++ b/test.pkg.all.sh
@@ -3,27 +3,9 @@
 set -exu  # Strict shell (w/o -o pipefail)
 
 distros="
-    el:8
-    fedora:25
-    fedora:26
-    fedora:27
-    fedora:28
-    fedora:29
-    fedora:30
-    fedora:31
-    debian:stretch
-    debian:buster
-    ubuntu:xenial
     ubuntu:bionic
-    ubuntu:disco
-    ubuntu:eoan
 "
 
-if ! type packpack; then
-    echo "Unable to find packpack"
-    exit 1
-fi
-
 for distro in $distros; do
     export OS="${distro%%:*}"
     export DIST="${distro#*:}"
@@ -31,9 +13,6 @@ for distro in $distros; do
         export OS=centos
     fi
 
-    rm -rf build
-    packpack
-
     docker run \
         --volume "$(realpath .):/tarantool-php" \
         --workdir /tarantool-php                \
diff --git a/test.pkg.sh b/test.pkg.sh
index 3c1c271..e80d587 100755
--- a/test.pkg.sh
+++ b/test.pkg.sh
@@ -29,8 +29,14 @@ fi
 # Update available packages list.
 ${PM_SYNC}
 
+# Verify against php7.4 on Ubuntu Bionic.
+apt-get -y install software-properties-common
+add-apt-repository -y ppa:ondrej/php
+apt-get update > /dev/null
+
 # Install php interpreter.
-${PM_INSTALL_NAME} php
+apt-get install -y php7.4
+php --version
 
 # Install php-tarantool package.
 ${PM_INSTALL_FILE} build/${PKG_GLOB}

Steps:

$ OS=ubuntu DIST=bionic /path/to/packpack/packpack # build
$ ./test.pkg.all.sh # verify

A package is in build/ directory, named like php7.4-tarantool_0.3.0.40-1_amd64.deb.

Now php-tarantool packages are deployed into two kinds of repositories: packagecloud.io and to S3 based ones. The download.tarantool.org now points to the latter, but we keep packagecloud.io for a while to don't break users, which use it directly (we don't recommend it though). See [1] for more info about deprecation of our packagecloud.io repositories. The deployment scripts allows to provide two configurations: production and staging. The former is for deployment from the master branch and from a git tag. The latter is for deployments from a developer branch. It is useful when something should be verified in a specific environment using a built package or when the deployment process itself was modified and should be tested. The difference between production and staging deployment process is how duplicate package versions are handled. It give an error for production deployment, but does not for staging. The new package is discarded in the case for packagecloud.io, but it replaces the old package in S3 repositories. Read comments in the deployment scripts for more details. [1]: tarantool/tarantool#4947 Fixes #117

Totktonada · 2020-06-29T21:46:59Z

Implemented in PR #162 and merged into php7-v2 branch. The packages will actually land into our yum / apt repositories after #137 (I plan to do it today).

Totktonada · 2020-08-24T13:14:34Z

Build php-tarantool for CentOS 7 against remi-php71 repository:

diff --git a/rpm/prebuild.sh b/rpm/prebuild.sh
new file mode 100755
index 0000000..c4420af
--- /dev/null
+++ b/rpm/prebuild.sh
@@ -0,0 +1,11 @@
+#!/bin/sh
+
+set -exu  # Strict shell (w/o -o pipefail)
+
+# Preserve environment variables (just in case).
+SUDO="sudo -E"
+
+${SUDO} yum install -y yum-utils
+${SUDO} yum install -y https://rpms.remirepo.net/enterprise/remi-release-7.rpm
+${SUDO} yum-config-manager --enable remi-php71
+${SUDO} yum update -y
diff --git a/test.pkg.all.sh b/test.pkg.all.sh
index 138a1b0..2e89b3b 100755
--- a/test.pkg.all.sh
+++ b/test.pkg.all.sh
@@ -3,27 +3,9 @@
 set -exu  # Strict shell (w/o -o pipefail)
 
 distros="
-    el:8
-    fedora:25
-    fedora:26
-    fedora:27
-    fedora:28
-    fedora:29
-    fedora:30
-    fedora:31
-    debian:stretch
-    debian:buster
-    ubuntu:xenial
-    ubuntu:bionic
-    ubuntu:eoan
-    ubuntu:focal
+    el:7
 "
 
-if ! type packpack; then
-    echo "Unable to find packpack"
-    exit 1
-fi
-
 for distro in $distros; do
     export OS="${distro%%:*}"
     export DIST="${distro#*:}"
@@ -31,9 +13,6 @@ for distro in $distros; do
         export OS=centos
     fi
 
-    rm -rf build
-    packpack
-
     docker run \
         --volume "$(realpath .):/tarantool-php" \
         --workdir /tarantool-php                \
diff --git a/test.pkg.sh b/test.pkg.sh
index 3c1c271..5231ab5 100755
--- a/test.pkg.sh
+++ b/test.pkg.sh
@@ -29,8 +29,15 @@ fi
 # Update available packages list.
 ${PM_SYNC}
 
+# Verify against php7.1 on CentOS 7.
+yum install -y yum-utils
+yum install -y https://rpms.remirepo.net/enterprise/remi-release-7.rpm
+yum-config-manager --enable remi-php71
+yum update -y
+
 # Install php interpreter.
 ${PM_INSTALL_NAME} php
+php --version
 
 # Install php-tarantool package.
 ${PM_INSTALL_FILE} build/${PKG_GLOB}

The same for remi-php73:

diff --git a/rpm/prebuild.sh b/rpm/prebuild.sh
new file mode 100755
index 0000000..54359fc
--- /dev/null
+++ b/rpm/prebuild.sh
@@ -0,0 +1,11 @@
+#!/bin/sh
+
+set -exu  # Strict shell (w/o -o pipefail)
+
+# Preserve environment variables (just in case).
+SUDO="sudo -E"
+
+${SUDO} yum install -y yum-utils
+${SUDO} yum install -y https://rpms.remirepo.net/enterprise/remi-release-7.rpm
+${SUDO} yum-config-manager --enable remi-php73
+${SUDO} yum update -y
diff --git a/test.pkg.all.sh b/test.pkg.all.sh
index 138a1b0..2e89b3b 100755
--- a/test.pkg.all.sh
+++ b/test.pkg.all.sh
@@ -3,27 +3,9 @@
 set -exu  # Strict shell (w/o -o pipefail)
 
 distros="
-    el:8
-    fedora:25
-    fedora:26
-    fedora:27
-    fedora:28
-    fedora:29
-    fedora:30
-    fedora:31
-    debian:stretch
-    debian:buster
-    ubuntu:xenial
-    ubuntu:bionic
-    ubuntu:eoan
-    ubuntu:focal
+    el:7
 "
 
-if ! type packpack; then
-    echo "Unable to find packpack"
-    exit 1
-fi
-
 for distro in $distros; do
     export OS="${distro%%:*}"
     export DIST="${distro#*:}"
@@ -31,9 +13,6 @@ for distro in $distros; do
         export OS=centos
     fi
 
-    rm -rf build
-    packpack
-
     docker run \
         --volume "$(realpath .):/tarantool-php" \
         --workdir /tarantool-php                \
diff --git a/test.pkg.sh b/test.pkg.sh
index 3c1c271..1df9354 100755
--- a/test.pkg.sh
+++ b/test.pkg.sh
@@ -29,8 +29,15 @@ fi
 # Update available packages list.
 ${PM_SYNC}
 
+# Verify against php7.3 on CentOS 7.
+yum install -y yum-utils
+yum install -y https://rpms.remirepo.net/enterprise/remi-release-7.rpm
+yum-config-manager --enable remi-php73
+yum update -y
+
 # Install php interpreter.
 ${PM_INSTALL_NAME} php
+php --version
 
 # Install php-tarantool package.
 ${PM_INSTALL_FILE} build/${PKG_GLOB}

Extracted from #117 (comment)

This was referenced Sep 11, 2018

PHP7 continuous integration #136

Closed

Move php7-v2 to master #137

Closed

Totktonada mentioned this issue Dec 12, 2018

Return working tests on php7.0-php7.2 #140

Closed

Totktonada changed the title ~~PHP7 deb package~~ Deploy PHP7 Deb/RPM packages Dec 12, 2018

This was referenced Mar 23, 2020

Add upcoming PHP7 support #46

Closed

Enable extension assembly on OS with PHP7 (Jenkins) and fix PHP version in RPM/DEB specifications #116

Closed

Totktonada added this to the Support PHP 7.* milestone Mar 23, 2020

This was referenced Mar 24, 2020

packpack el8 - cannot install the best candidate for the job packpack/packpack#115

Closed

Revive PHP 7 connector #153

Merged

Totktonada added a commit that referenced this issue Mar 28, 2020

build/debian: enable the extension by default

f6e968f

I guess it is the behaviour, which is expected by a user: when an extension is installed it should be available for use. Part of #117

Totktonada added a commit that referenced this issue Mar 28, 2020

build/debian: enable the extension by default

40d8795

I guess it is the behaviour, which is expected by a user: when an extension is installed it should be available for use. Part of #117

Totktonada mentioned this issue Apr 28, 2020

infra: deprecation plan for packagecloud tarantool/tarantool#4947

Closed

Totktonada mentioned this issue Jun 24, 2020

travis-ci: deploy RPM / Deb packages #162

Merged

Totktonada self-assigned this Jun 29, 2020

Totktonada closed this as completed Jun 29, 2020

Totktonada added a commit that referenced this issue Apr 8, 2022

doc: describe how to build against ppa:ondrej/php

68887a7

Extracted from #117 (comment)

Totktonada added a commit that referenced this issue Apr 8, 2022

doc: describe how to build against remi repository

5a88272

Extracted from #117 (comment)

Totktonada mentioned this issue Apr 11, 2022

Update driver to PHP8.1 #171

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deploy PHP7 Deb/RPM packages #117

Deploy PHP7 Deb/RPM packages #117

cyanide-burnout commented Apr 16, 2017

Totktonada commented Dec 12, 2018

Totktonada commented Jun 2, 2020

Totktonada commented Jun 29, 2020

Totktonada commented Aug 24, 2020

Deploy PHP7 Deb/RPM packages #117

Deploy PHP7 Deb/RPM packages #117

Comments

cyanide-burnout commented Apr 16, 2017

Totktonada commented Dec 12, 2018

Totktonada commented Jun 2, 2020

Totktonada commented Jun 29, 2020

Totktonada commented Aug 24, 2020