Skip to content

Latest commit

 

History

History
872 lines (513 loc) · 16.6 KB

REFERENCE.md

File metadata and controls

872 lines (513 loc) · 16.6 KB
Raw

Reference

Table of Contents

Classes

Defined types

Resource types

Classes

clickhouse

Top level class for ClickHouse DBMS installation and management.

Examples

Simple use
include clickhouse
Install server and client
class { 'clickhouse':
  server => true,
}
Install everything and manage repository
class { 'clickhouse':
  server      => true,
  client      => true,
  manage_repo => true,
}

Parameters

The following parameters are available in the clickhouse class.

server

Data type: Boolean

Is clickhouse-server should be installed or not. It won't be installed by default, because some time you need the client only.

Default value: false

client

Data type: Boolean

Is clickhouse-client should be installed or not.

Default value: true

manage_repo

Data type: Boolean

Is apt or yum repository should be managed. Set to false by default to not affect your own repositories policy.

Default value: false

user

Data type: String[1]

User for configs owning. Strongly recommended staying default.

It's highly unrecommended to change default user, but feel free to shot in a knee. You have to adjust as well the data directory and the whole configs everywhere.

Default value: 'clickhouse'

group

Data type: String[1]

Group for configs, see user parameter.

Default value: $user

clickhouse::client

The class installs clickhouse-client.

Examples

Simple use
include clickhouse::client
Use with params
class { 'clickhouse::client':
  package_name   => 'clickhouse-client-custom',
  package_ensure => 'hold',
  user           => 'custom-user',
  config_d_dir   => '/some/path',
}

Parameters

The following parameters are available in the clickhouse::client class.

package_name

Data type: String[1]

Package to be installed.

Default value: 'clickhouse-client'

package_ensure

Data type: String[1]

Client package ensure. See ensure attribute for package resource.

Default value: 'installed'

conf_d_dir

Data type: Optional[Stdlib::Unixpath]

Deprecated, use config_d_dir.

Default value: undef

config_d_dir

Data type: Stdlib::Unixpath

Directory for custom configs. Unmanaged configs will be removed from the dirrectory during puppet running.

Default value: $conf_d_dir

clickhouse::repo

Installs yum or apt repository for ClickHouse DBMS.

Uses Altinity repository for yum and yandex for apt.

Examples

Simple use
include clickhouse::repo

clickhouse::server

The class installs ClickHouse server and manages service

Examples

Simple use
include clickhouse::server
Use with params
class { 'clickhouse::server':
  package_name   => 'clickhouse-server-custom',
  package_ensure => 'latest',
  service_name   => 'clickhouse-server',
  service_ensure => false,
  service_enable => false,
}

Parameters

The following parameters are available in the clickhouse::server class.

package_name

Data type: String[1]

Server package to be installed.

Default value: 'clickhouse-server'

package_ensure

Data type: String[1]

Server package ensure. See ensure attribute for package resource.

Default value: 'installed'

service_name

Data type: String[1]

Name of the managed service for clickhouse-server.

Default value: $package_name

service_ensure

Data type: Variant[Boolean, Enum[ 'running', 'stopped' ]]

Desired state for $service_name, see ensure for service resource.

Default value: 'running'

service_enable

Data type: Variant[Boolean, Enum[ 'manual', 'mask' ]]

If $service_name should be enabled, see enable for service resource.

Default value: true

config_service_notify

Data type: Boolean

If true, every config managed by this module and requires for server restart will trigger service refresh.

Default value: true

conf_dir

Data type: Optional[Stdlib::Unixpath]

Deprecated, use config_dir

Default value: undef

config_dir

Data type: Stdlib::Unixpath

Directory with clickhouse-server configuration.

Default value: $conf_dir

conf_d_dir

Data type: Optional[Stdlib::Unixpath]

Deprecated, use config_d_dir

Default value: undef

config_d_dir

Data type: Stdlib::Unixpath

Directory with clickhouse-server included configuration. Unmanaged configs will be removed from the directory during puppet run.

Default value: $conf_d_dir

users_d_dir

Data type: Stdlib::Unixpath

Directory with clickhouse-server users configuration. Unmanaged configs will be removed from the directory during puppet run.

Default value: "${config_dir}/users.d"

clickhouse::server::config::default_localhost

clickhouse::server::config::default_localhost By default, ClickHouse listens only localhost and user default is able to connect from anywhere. This class, if included, restricts network access.

Examples

Simple use
include clickhouse::server::config::default_localhost

clickhouse::server::config::memory

clickhouse::server::config::memory This class provide basic memory adjustment for keep ClickHouse server in reasonable limits

Examples

Simple use
include clickhouse::server::config::memory

Parameters

The following parameters are available in the clickhouse::server::config::memory class.

reserved_memory

Data type: Integer[0]

Memory to leave for system usage and CH overheads like merges and mutations

Default value: 4294967296

mark_cache_size

Data type: Integer[0]

See CH documentation

Default value: 5368709120

uncompressed_cache_size

Data type: Optional[Integer[0]]

See CH documentation

Default value: undef

max_memory_usage_for_all_queries

Data type: Optional[Integer[0]]

See CH documentation

Default value: undef

max_memory_usage

Data type: Optional[Integer[0]]

See CH documentation

Default value: undef

use_uncompressed_cache

Data type: Boolean

See CH documentation

Default value: true

external_group_by

Data type: Boolean

See CH documentation

Default value: true

external_sort

Data type: Boolean

See CH documentation

Default value: true

memory_check

Data type: Boolean

Check if class has proper parameters and raise errors otherwise

Default value: true

service_notify

Data type: Boolean

If ClickHouse server should be restarted on the config update

Default value: $clickhouse::server::config_service_notify

clickhouse::server::config::zookeeper

clickhouse::server::config::zookeeper Set proper zookeeper config

Examples

Use with params
class { 'clickhouse::server::config::zookeeper':
    nodes    => { 'server1' => 1},
    user     => 'user',
    password => 'password',
}

Parameters

The following parameters are available in the clickhouse::server::config::zookeeper class.

nodes

Data type: Hash[String[1], Integer[1, 255]]

hash of zk_server: id to define nodes

port

Data type: Integer[1, 65536]

cilent port of zookeeper cluster

Default value: 2181

session_timeout_ms

Data type: Integer[1]

maximum timeout for client session in milliseconds

Default value: 30000

operation_timeout_ms

Data type: Integer[1]

maximum timeout for operation in milliseconds

Default value: 10000

root

Data type: Optional[Stdlib::Unixpath]

ZNode, that is used as root for znodes used by ClickHouse server

Default value: undef

user

Data type: Optional[String[1]]

user if zookeeper uses authorization

Default value: undef

password

Data type: Optional[String[1]]

password if zookeeper uses authorization

Default value: undef

mode

Data type: String[1]

zookeeper nodes, ports and optional user and password are confidential data and normally should not be readable

Default value: '0440'

service_notify

Data type: Boolean

If ClickHouse server should be restarted on the config update

Default value: $clickhouse::server::config_service_notify

Defined types

clickhouse::client::config

This type creates custom configuration files for clickhouse-client.

Examples

Usage
clickhouse::client::config { 'prompt':
  data => {'prompt_by_server_display_name' => [{
      'experimental' => ['{display_name} \x01\e[1;35m\x02:)\x01\e[0m\x02 '],
  }]},
}
#
# Will create file `/etc/clickhouse-client/conf.d/prompt.xml`:
# <config>
#   <prompt_by_server_display_name>
#     <experimental>{display_name} \x01\e[1;35m\x02:)\x01\e[0m\x02 </experimental>
#   </prompt_by_server_display_name>
# </config>

Parameters

The following parameters are available in the clickhouse::client::config defined type.

data

Data type: Hash

This hash will be converted into xml config placed in $clickhouse::client::config_d_dir.

Root will be <config> by default.

ensure

Data type: Enum[ 'present', 'file', 'absent' ]

Subset of attribute ensure for file resource.

Default value: 'present'

mode

Data type: String[1]

Desired permissions mode for the config file, see mode attribute for file resource.

Default value: '0644'

clickhouse::error

clickhouse::error Implements error which cause 6 exit code with --detailed-exitcodes argument for puppet run. Work around notifying about severe problem

Examples

Simple use
clickhouse::error { 'Error message':
}

clickhouse::server::config

This type creates custom configuration files for clickhouse-server.

Examples

Usage
clickhouse::server::config { 'macros':
  data    => {'macros' => [
      {'hostname' => ['host.domain.TLD'],}
  ]},
  section => 'config',
}
#
# Will create file `/etc/clickhouse-server/conf.d/macros.xml`:
# <yandex>
#   <macros>
#     <hostname>host.domain.TLD</hostname>
#   </macros>
# </yandex>

Parameters

The following parameters are available in the clickhouse::server::config defined type.

data

Data type: Hash

This hash will be converted into xml config placed in $clickhouse::server::config_d_dir.

Root will be <yandex> by default.

section

Data type: Enum['config', 'users']

Should the file be created in configs or users include directory.

ensure

Data type: Enum[ 'present', 'file', 'absent' ]

Subset of attribute ensure for file type.

Default value: 'present'

mode

Data type: String[1]

Desired permissions mode for the config file, see mode attribute for file resource.

Default value: '0644'

service_notify

Data type: Boolean

If service should be restarted on the config changing.

Default value: false

clickhouse::server::config::profile

This type defines the ClickHouse profile with specified options

Examples

clickhouse::server::config::profile { 'profile_name':
    databases => {
        'db_name' => {
            'table_name'         => ['filter'],
            'another_table_name' => ['another filter'],
        },
    },
}

Parameters

The following parameters are available in the clickhouse::server::config::profile defined type.

profile

Data type: String[1]

User profile

Default value: $title

settings

Data type: Hash

Profile settings

clickhouse::server::config::user

This type defines the ClickHouse users with specified parameters

Examples

clickhouse::server::config::user { 'username':
    network   => {
        'ip'          => [
            '::',
            '0.0.0.0',
        ],
        'host'        => [
            'host1.local',
            'host2.local',
        ],
        'host_regexp' => [
            '[^.]*\.domain\.TLD',
        ],
    },
    profile   => 'profile_name',
    quota     => 'quota_name',
    password  => 'password',
    databases => {
        'db_name' => {
            'table_name'         => ['filter'],
            'another_table_name' => ['another filter'],
        },
    },
}

Parameters

The following parameters are available in the clickhouse::server::config::user defined type.

networks

Data type: Struct[{ Optional[ip] => Array[String[1], 1], Optional[host] => Array[String[1], 1], Optional[host_regexp] => Array[String[1], 1], }]

Defines network nodes where from user could connect

profile

Data type: String[1]

User profile

Default value: 'default'

quota

Data type: String[1]

User quota

Default value: 'default'

user

Data type: String[1]

User name

Default value: $title

password

Data type: String[0]

Plaintext password. Will be hashed into sha256 format in xml file

Default value: ''

password_sha256

Data type: Optional[ Pattern[/\A[0-9a-fA-F]{64}\Z/] ]

Optional parameter. If defined, $password must not be defined

Default value: undef

databases

Data type: Optional[Hash]

Optional hash of allowed DBs, tables and filters to implement ACLs

Default value: undef

allow_databases

Data type: Optional[Array[String[1]]]

Optional parameter to restrict access to specified databases

Default value: undef

allow_dictionaries

Data type: Optional[Array[String[1]]]

Optional parameter to restrict access to specified dictionaries

Default value: undef

Resource types

clickhouse_database

Manages databases on ClickHouse server

Properties

The following properties are available in the clickhouse_database type.

engine

Valid values: Lazy, MySQL, Ordinary

Engine of the database

Default value: Ordinary

ensure

Valid values: present, absent

CREATE or DROP DATABASE

Default value: present

Parameters

The following parameters are available in the clickhouse_database type.

engine_settings

This parameter depends on the engine parameter: Lazy: must be Integer[1] MySQL: must be Array[String, 4, 4] Ordinary: must be undefined

See: https://clickhouse.tech/docs/en/engines/database_engines/

force

Valid values: true, false, yes, no

DROP database even if it contains tables

If ENGINE=MySQL for an existing database, it will be dropped anyway since it's just a connector

Default value: false

name

namevar

Name of the database

provider

The specific backend to use for this clickhouse_database resource. You will seldom need to specify this --- Puppet will usually discover the appropriate provider for your platform.