The "PHP 7.1 install from hell" on Debian 9

[hostingnuggets]

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 4.8.2
• Ruby: 2.3.3
• Distribution: Debian 9.4
• Module version: 5.3.0

How to reproduce (e.g Puppet code you use)

manifest:

class { '::php': }

node hiera YAML file:

php::globals::php_version: '7.1'
php::manage_repos: true
php::extensions:
  curl: {}
  gd: {}
  intl: {}
  mcrypt: {}
  mysql: { so_name: 'mysqlnd' }
  xml: {}

What are you seeing

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Operator '[]' is not applicable to an Undef Value. at /etc/puppet/code/environments/production/modules/php/manifests/repo/debian.pp:89:19 on node <REMOVED>

Error: Failed to apply catalog: Found 1 dependency cycle:
(Exec[apt_update] => Class[Apt::Update] => Package[lsb-release] => Apt::Source[source_php_71] => Apt::Setting[list-source_php_71] => File[/etc/apt/sources.list.d/source_php_71.list] => Class[Apt::Update] => Exec[apt_update])

Error: Failed to apply catalog: Found 1 dependency cycle:
(Exec[apt_update] => Package[lsb-release] => Apt::Source[source_php_71] => Apt::Setting[list-source_php_71] => File[/etc/apt/sources.list.d/source_php_71.list] => Class[Apt::Update] => Exec[apt_update])

What behaviour did you expect instead

PHP version 7.1 to be installed without any issues/errors/warnings from the sury.org Debian APT repository.

Output log

See above in "What are you seeing"

Any additional information you'd like to impart

I can't seem to find any actual documentation on how to install a different version of PHP on Debian 9.

[bastelfreak]

Hi @hostingnuggets, have you tested against our master branch or the latest release as well? I think on debian 9 we currently only support the default php version.

[hostingnuggets]

@bastelfreak The debian repo code https://github.com/voxpupuli/puppet-php/blob/master/manifests/repo/debian.pp has not changed since 3 months so there is no point for me testing with the master branch if the relevant code has not been fixed...

It would be nice to see in the documentation which version of PHP is supported by which distribution. But then on Debian 9 if you only support the default PHP version why is there code to support the sury repository? or is that for Debian 8 only?

[bastelfreak]

so I took a look at the code in the 5.3.0 release:

puppet-php/manifests/repo/debian.pp

Line 89 in 4d99a12

release => $facts['os']['distro']['codename'],

I'm not sure why that fails for you, it might be related to the EOL puppet version. Can you please test the latest release with a modern puppet release? 4.10 is the oldest version we support and puppet 5 is recommended.

[hostingnuggets]

@bastelfreak as far as I know puppet version 4.8.2 is not EOL yet. Could you please cite the source where you base the fact that puppet 4.8.2 would already be EOL?

Unfortunately I am using the latest Debian 9.5 release which ships with puppet 4.8.2 so I have to use that specific version by policy. Do you realize that by limiting the compatibility of this module to puppet version >= 4.10.0 you are excluding every Debian 9 users? In my opinion you should consider removing Debian 8 and Debian 9 support from the supported operating systems in this module's metadata.json file because it is not entirely true.

[bastelfreak]

We don't exclude Debian 9 users. Puppetlabs provides apt repos: https://apt.puppetlabs.com/
They provide you the latest Puppet 4 and also Puppet 5. I don't have the source by hand, but I do know that Puppet 4.10 is the first version that properly supports hiera5 for data in modules, which is something that we use in our modules. Please also keep in mind that Puppet 4 will be completely EOL by the end of the year, so we will probably drop Puppet 4 entirely in January.

[hostingnuggets]

@bastelfreak well you are at least excluding me because for policy reasons I must use the official packages from my Linux distribution, in my case Puppet 4.8.2 from the Debian 9 official APT package repository.

Anyway let's not focus on my specific puppet version because even if I would have puppet version 5 that will not automagically fix https://github.com/voxpupuli/puppet-php/blob/master/manifests/repo/debian.pp This manifest file is in my opinion missing good logic and does not work properly as you can end up with cases such as having PHP 5.6 from dotdeb installed at the same time as with PHP 7.1 (or another version). See my issue #458 for more details.

Based on other Debian issues I have the feeling here that this puppet module is neglecting Debian users. It would be nice if some improvement could be done here. Also please do not exclude people just because they have to use soon-to-be-EOL packages.

Thank you.

[bastelfreak]

So a few things here:
Vox Pupuli has no influence in the Puppet versions shipped in any Linux operating system or in any company policy. We Do not want to encourage our users to use outdated software. That is why we do not have Debian 7 in our metadata.json, even if it might still work, it is just EOL and should not be used. Puppet 4 is soon EOL as well. We can't support Puppet <4.10 in our organisation because it has no data-in-modules support with hiera 5. We do not introduce new features/breaking changes if users of up2date operating systems can't use them. Puppet created the apt/yum repos for this usecase. It allows users to use a modern Puppet version. That still has support. As far as I know 4.8.X is deprecated upstream and does not receive any securitypatches. It is simply not possible to test all operating systems with all available puppet versions on all of our 100+ modules. We offer a solution that technically works for all non-EOL operating systems, documented in the metadata.json files.

We do not neglect any operating system. All contributors do this as a hobby and we try our best to achieve a high quality of all modules. There is a test that proves that this module works on debian 9 with the default php version: https://github.com/voxpupuli/puppet-php/blob/master/spec/acceptance/php_spec.rb

There are always edges cases on each platform that are not tested or not documented or simply do not work. This is an open source project and we always want to improve our modules. Patches are always highly appreciated.

[hostingnuggets]

@bastelfreak thank you for taking time to explain. You mention here again that as far as you know puppet 4.8.x is deprecated but I can not find anywhere any information which clearly states that it is deprecated. Could you please state the source (URL) where you base the fact that puppet 4.8.x is deprecated?

[ekohl]

Unfortunately I am using the latest Debian 9.5 release which ships with puppet 4.8.2 so I have to use that specific version by policy. Do you realize that by limiting the compatibility of this module to puppet version >= 4.10.0 you are excluding every Debian 9 users? In my opinion you should consider removing Debian 8 and Debian 9 support from the supported operating systems in this module's metadata.json file because it is not entirely true.

I disagree with this. It will run on Debian 9 but we also specify you need Puppet 4.10.0.

We have a debate on this in https://groups.io/g/voxpupuli/topic/get_rid_of_puppet4/15908402?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,15908402 and it was decided that we as a group only want to support 4.10.0 and higher. This isn't to exclude users but simply a matter of time and effort. We are all volunteers and have limited time available. A smaller testing matrix helps with that. As it is now I think we already have plenty of open issues and PRs.

If this is important to you, I welcome you to become a member. We can look at a way to support your environment while not taking up too much time of other members. For example by running acceptance tests on Debian 9 with OS packages. IMHO we can decide this on a per-module basis.

The way I look at Open Source is that it allows us to fix our issues but that does mean you often have to fix it yourself.

[hostingnuggets]

@ekohl I would really like to see this module support Debian 9 out-of-the-box with the Debian 9 included puppet package. So as I am no developer I can contribute differently and would be pleased to set you up a VPS with Debian 9 including Debian 9 with its own official puppet 4.8.2 package where you could run your acceptance tests in order to make things work out-of-the-box with Debian 9.

[bastelfreak]

After a lot of digging around:

• Puppet enterprise 2016.5.2 contains Puppet 4.8.2
• It was released on 2017-02-07
• It is EOL at least since 2018-02-21

As far as I know the separate support overview for FOSS components got removed.

---

Besides my work for Vox Pupuli, I am a DevOps engineer at a big hosting company with several thousand servers. From time to time different operating system maintainer claim to support a specific version of a software and promise patches. But where are they coming from, when upsteam simply does not deliver such patches? In my personal opinion, I cannot rely on those packages for my infrastructure. I also understand that other DevOps/Ops people have a different view on this.

---

Vox Pupuli cannot test each Puppet/facter/ruby version that any operating system claims to support. The testmatrix would we way too huge, somebody would need to pay the CI resources for this and we would need more manpower to handle all the edge cases. And for what purpose? I do not think that Puppet Inc would fix bugs in EOL software and I, as a maintainer, want to move on with the codebase and use the new features of the language. We deliver software to users and I still do not think that we should encourage them to run legacy/outdated software. We are always happy to give tips/help with the upgrade of legacy infrastructures into a more modern and up2date environment.

[ekohl]

So as I am no developer I can contribute differently and would be pleased to set you up a VPS with Debian 9 including Debian 9 with its own official puppet 4.8.2 package where you could run your acceptance tests in order to make things work out-of-the-box with Debian 9.

The problem isn't access to these environments. We can easily use containers and I'm sure there's some way to get beaker to use OS packages. Travis runs docker containers. The problem is time.

We're all volunteers with limited time available that maintain the parts that are important to us. For most it's because the company that pays their salary depends on parts of it. At least for me I'm involved because our product includes VP modules. To ensure they suit our use case, I invest time in those modules. I spend time on some modules we don't use to help other maintainers. This means in the future they are more likely to make time to help me. That's my view of how open source works.

In open source people the rule of thumb is that if something is important to you, you write the code. If you don't know how to write it, you invest time to learn it. Companies have the option to find a vendor that supports their needs. Other ways are consultants, hiring new employees with the knowledge or educating an existing employees.

Remember that the alternative is not doing it at all or writing it all in house. Open source is usually a much cheaper solution, but it's not free (as in money).

[dhollinger]

@hostingnuggets as @ekohl and @bastelfreak have already mentioned. We are all volunteers here with separate jobs and responsibilities to take care of. As such, we don't have the manpower to cover every potential release of puppet that someone may be on, hence we limit our support range to the minimum version supported by the latest supported point release of Puppet Enterprise (which is currently >= 4.10.0).

If we start supporting Debian 9's Puppet version, then on principal we would also have to support the versions of puppet included in Debian 8, CentOS/RHEL/OEL 6 and 7, Ubuntu 16.04 and 18.04, etc. That is not viable and most of the listed distributions come packaged with an EoL'd version of Puppet anyway. Our recommended pattern for supported puppet versions it to stick with the Officially supported installation method from the Puppet project, which is their repositories. I know distribution purists (myself included) prefer to use system packages, but the OS repos simply don't keep up with software these days and are all over the place with their package selection that we simply cannot support installation methods outside of the recommended installation paths.

I know this sucks and may not be ideal, I am truly sorry, however this is no different than how other Open Source projects (like Gitlab and CentOS for example) operate. If you need extended support on software, there's always the paid versions (Gitlab EE and RHEL for the above examples).

We are more than willing to work with you on the process of moving to a newer version of Puppet or help point you to a working version of the PHP module for your puppet version. Feel free to fork and maintain yourself as well (we do take PRs). We just can't increase our support range for a single distribution.

[ekohl]

Another option is to ask Debian to include a newer version (for example 4.10.10 or even a 5.x version) in stretch-backports.

[hostingnuggets]

Thank you all for the interesting discussion around this problem and I understand the point that you should not support EOL versions. I was just hoping that this puppet module could support Debian 9 out-of-the-box by not having to install packages from an external/foreign apt source repository.

[RedChops]

As a reply to #459 (comment), I'm also running the old version of Puppet 4.8.2 on my machines (No reason for me to update to 4.10, I'm evaluating a jump to 5x but I'm using Foreman so it's a "process"). I don't have that fact on any of my machines. What I do have that matches is $facts['os']['lsb']['distcodename']. Changing https://github.com/voxpupuli/puppet-php/blob/master/manifests/repo/debian.pp#L87
to that variable instead allowed the code to work fine. I don't know enough about the changes to later versions of Puppet to say if this is a good fix or not, but it did work for me.

[ekohl]

Changing https://github.com/voxpupuli/puppet-php/blob/master/manifests/repo/debian.pp#L87
to that variable instead allowed the code to work fine.

We could even remove the line since puppetlabs-apt defaults to that value (though by the old $facts['lsbdistcodename'] which should be even more compatible. I'd be happy with that and would approve such a PR.

Personally I've been very reluctant to change these facts where the old ones sufficed exactly for the compatibility. In some places the new facts are superior (structured networking facts are much easier to parse), other than that I've been sticking to old facts.

[dhollinger]

@ekohl I think this will have to come down to a discussions within the community. I was under the impression that old-style facts are deprecated and could be removed at any moment if Puppet so chooses to do so. Our immediate concern is to maintain compatibility with our supported version range.

I personally am against reverting code to support unsupported versions. It encourages bad practice, regardless of the reality of a organization's situation. We are short-handed as it is and adding another thing we have to watch for (The eventual removal of Facter 2.x facts) is unnecessary burden for a handful of users.

[ekohl]

Passing in a variable when the default suffices and actually breaks support for some users helps nobody. #494 is untested, but it should be more compatible by relying on puppetlabs-apt for more features.

[ekohl]

With #494 we've merged improvements. With that and the above discussion I think we can close this. It's clear we don't want to officially support Puppet 4.8.2 on Debian but if there are concrete issues with an easy middle ground or (maintainable) patches then we're happy to accept them.

In this case the very concrete suggestion from @RedChops made it easy. It's a great example of the open source approach mentioned earlier. Looking back, #459 (comment) should have been enough of a hint already.

[hostingnuggets]

@ekohl finally some good sense, thank you very much for supporting Debian with out-of-the-box puppet version 4.8.x 👍

[bastelfreak]

A subtle but important difference: It is compatible, but not supported. There is no guarantee that it will keep working in the future because there are no tests for it.