cfpuppetserver

Description

The package does standard procedure of installing Puppet Server, Puppet DB, PostgreSQL, r10k, librarian-puppet and making it work all togather. It also lives in peace with cfsystem and cfnetwork packages.

Environment configuration

The configurations expects you to provide Hiera version 4 configuration in environments. Example can be taken from codingfuture/puppe-test.

NOTE1: there is a known closed bug in puppet <=4.3.2 - please make sure that all Hiera hierarchy files exist and both empty YAML and JSON files include '{}' at least

VERY IMPORTANT!!!

Now, the modules uses [cfdb][] for High Availability support out-of-the-box. The consequences is that setup process is quite tricky - we need some facts and resources to be populated into Puppet DB while Puppet DB is malfunctioning until the stack is fully configured. Most likely, you will see some errors during conversion process and both Puppet DB & Puppet Server stop functioning.

In that case, you can continue re-provisioning previously compiled catalog until Puppet Server can continue compiling new catalogs with the following command:

/opt/puppetlabs/bin/puppet apply --catalog /opt/puppetlabs/puppet/cache/client_data/catalog/$(/bin/hostname --fqdn).json
/opt/puppetlabs/bin/puppet agent --test

Upgrade to puppetserver >= 2.5.0

As there is incompatible change related to bootstrap.cfg, please use cfpuppetserver < v0.10 for puppetserver < 2.5.

Upgrade procedures:

• Update to cfpuppetserver >= v0.10
• Manually deploy to current Puppet servers: puppet agent -t
• Puppet Server will fail to restart in 180 seconds
• Upgrade puppetserver/puppetdb/puppet-agent packages to latest versions
• Manually start services:
  • /bin/systemctl stop cfpuppetdb.service cfpuppetserver.service
  • /bin/systemctl start cfpuppetdb.service cfpuppetserver.service
• Wait for services to startup monitoring netstat -pletn
• Try Puppet deployment

Global Hiera config

Puppet 4 has own implementation of lookup() which goes through:

• Global Hiera
• Per-environment Data Providers (Hiera, custom function)
• Per-module Data Providers (Hiera, custom function)

You should not use global Hiera any more. All configurations should be set in environments as mentioned above.

Global Hiera config is as follows:

---
:backends:
  - yaml
:hierarchy:
  - global
:yaml:
  # Make sure to use hiera.yaml in environments
  :datadir: "/etc/puppetlabs/code/hieradata"

Adding new Puppet clients

This module also provides a handy tool to initalize new puppet client hosts:

~#  /opt/codingfuture/bin/cf_gen_puppet_client_init
Usage: cf_gen_puppet_client_init <certname> [<cflocation> [<cflocationpool> [<http_proxy>]]

Manual (re-)deployment of Puppet environments

~# /opt/codingfuture/bin/cf_r10k_deploy

Automatic deployment via VCS (git) hook

~# ssh deploypuppet@puppet.example.com sudo /opt/codingfuture/bin/cf_r10k_deploy

Manual fast refresh of PuppetServer JRuby instances

~# /opt/codingfuture/bin/cf_puppetserver_reload

Manual purge of node trails from infrastructure

~# /opt/codingfuture/bin/cf_purge_node

Restoring default node deactivation and purging behavior

Auto-configuration of CF modules heavily depends PuppetDB info. To protect from accident disasters, automatic node info purging is disabled on purpose. It can be restored:

cfpuppetserver::puppetdb::settings_tune:
    database:
        'node-ttl': '7d'
        'node-purge-ttl': '14d'

Setup

Initial Puppet Server infrastructure

Either do manually (preferred for self-education) or use bundled setup script:

~# ./setup_puppetserver.sh
Usage: ./setup_puppetserver.sh <r10k_repo_url> [<certname=hostname> [<cflocation> [<cflocationpool> [<http_proxy=$http_proxy>] ] ] ]

Config for Puppet Server node

Up to date installation instructions are available in Puppet Forge: https://forge.puppet.com/codingfuture/cfpuppetserver

Please use librarian-puppet to deal with dependencies. If this module is used for server setup then librarian-puppet is installed automatically.

There is a known r10k issue RK-3 which prevents automatic dependencies of dependencies installation.

Examples

Please check codingufuture/puppet-test for example of a complete infrastructure configuration and Vagrant provisioning.

cfpuppetserver class

• deployuser = 'deploypuppet' - user name for auto deploy user for VCS hook
• deployuser_auth_keys = undef - list of ssh_authorized_keys configurations
• repo_url = undef - repository location in URI format (e.g. ssh://user@host/repo or file:///some/path)
• puppetserver = true - if true then assume Puppet Server lives on this host (affects firewall)
• puppetdb = true - if true then assume Puppet DB lives on this host (affects firewall)
• postgresql = true - if true then PostgreSQL is setup on this node
• autodiscovery = true - if true then other instances of PuppetDB & PuppetServers are automatically discovered to be added for PuppetDB access / server_urls
• puppetdb_hosts = [] - statically provide PuppetDB hosts (for autodiscovery = false)
• puppetserver_hosts = [] - statically provide PuppetServer hosts (for autodiscovery = false)
• client_hosts = [] - client hosts to allow access (dynamic ipset cfpuppet_clients)
• iface = 'any' - cfnetwork::iface name to listen for incoming client connections
• cluster = 'cfpuppet' - cfdb cluster to use
• database = 'puppetdb' - cfdb::database` to use in cluster
• is_cluster = false - goes directly to cfdb::instance
• is_secondary = false - goes directly to cfdb::instance
• is_arbitrator = false - goes directly to cfdb::instance
• allow_update_check = false - open firewall to connect to updates.puppetlabs.com, if enabled

cfpuppetserver::postgresql class

NOTE: if PostgreSQL is setup through this module then you SHOULD NOT setup other cfdb instances on the same node.

• $settings_tune = {} - goes directly to cfdb::instance
• $port = 5432 - goes directly to cfdb::instance
• $node_id = undef - required, if node ID cannot be retrieved from hostname in cluster mode
• $password = undef - force specific password instead of random generated
• $version = undef - version to pass to cfdb::postgresql
• $memory_weight = 200 - goes directly to cfdb::instance
• $memory_max = undef - goes directly to cfdb::instance
• $cpu_weight = 200 - goes directly to cfdb::instance
• $io_weight = 200 - goes directly to cfdb::instance
• $init_db_from = undef - overrides default init_db_from, if set

cfpuppetserver::puppetdb class

• $use_proxy = 'secure' - by default TLS channel is used for remote PostgreSQL connections. See cfdb::access.
• $port = 8081 - port to use for PuppetDB instance
• $max_connections = 30 - maximum number of connections per pool (there are two pools)
• $memory_weight = 100 - relative weight for auto-distribution of memory resources
• $memory_max = 256 - max memory in MB
• $cpu_weight = 100 - relative weight for auto-distribution of CPU resources
• $io_weight = 100 - relative weight for auto-distribution of I/O resources
• $cert_whitelist = [] - specify the CNs of Puppet PKI to be accepted. If not set:
  • if Puppet Server runs the same node then [$fqdn]
  • otherwise, all nodes with Puppet Server configured
• $settings_tune = {} - a tree structure of PuppetDB INI for fine control

cfpuppetserver::puppetserver class

• $autosign = false - DO NOT use in production. Enable auto-sign of client certificates.
• $global_hiera_config = 'cfpuppetserver/hiera.yaml' - default global Hiera config
• $memory_weight = 100 - relative weight for auto-distribution of memory resources
• $memory_max = undef - max memory in MB
• $cpu_weight = 100 - relative weight for auto-distribution of CPU resources
• $io_weight = 100 - relative weight for auto-distribution of I/O resources
• $activesupport_ver = '4.2.7.1' - version of activesupport gem to install
• $strict = 'warning' - goes directly to puppet.conf
• $disable_warnings = 'deprecations' - goes directly to puppet.conf
• $settings_tune = {} - optional tune, tree structure
  • 'puppetserver'
    • 'max-active-instances' = $::facts['processorcount'] + 1
    • 'max-requests-per-instance' = 1000
    • 'compile-mode' = 'off'
    • 'connect-timeout-milliseconds' = 15000
    • 'idle-timeout-milliseconds' = 600000