Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry-pick #20576 to 7.x: Build docker images based on Red Hat UBI #20753

Merged
merged 1 commit into from
Aug 24, 2020
Merged

Cherry-pick #20576 to 7.x: Build docker images based on Red Hat UBI #20753

merged 1 commit into from
Aug 24, 2020

Conversation

jsoriano
Copy link
Member

Cherry-pick of PR #20576 to 7.x branch. Original message:

What does this PR do?

Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images.

Additional checks have been added to packaging tests for labels and licenses.

Additional changes done to support it also in Elastic Agent images:

NOTE: Some of the changes for Elastic Agent could be actually moved to separate PRs, but they are quite intertwined and I would like to test them together. I decided to do them on this PR to don't have to conditionally disable package tests, and to be able to test everything together. But happy to move them to other PRs if preferred.

Why is it important?

To build images that can be certified for Red Hat OpenShift.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • Move licenses to /licenses
    • Add the proper licenses to x-pack and OSS images
  • Add labels
  • Add test to check that licenses are in place
  • Add test to check that required labels are present
  • Add build for Elastic Agent

Possibly to be done in a follow up PR

  • Refactor how files are prepared and copied, so it is not needed to change permissions in Dockerfile
  • Unify Dockerfiles for Agent and the rest of beats, as they are quite similar
  • Add checks for the included packages in Agent image

How to test this PR locally

Related issues

@jsoriano
Build docker images based on Red Hat UBI (elastic#20576)
d7acf80 
Add an additional docker build that builds images based on Red Hat UBI, following
Red Hat requirements for certified images.
Additional checks have been added to packaging tests for labels and licenses.
Additional changes done to support it also in Elastic Agent images:
* Home directory is prepared in a different stage (elastic#20356).
* Allow the docker image to be run with random user ids (elastic#18873).
* Explicitly select a Dockerfile and entry point template.
* Add NOTICE.txt file to all agent packages.
* Actually run package tests after building packages, added flag to allow root user.
* Improved checks on required packages, so they are not re-built if they already are.

(cherry picked from commit e31794d)
@jsoriano jsoriano added [zube]: In Review backport Team:Ingest Management Team:Integrations Label for the Integrations team Team:Platforms Label for the Integrations - Platforms team labels Aug 24, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Team:Ingest Management)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Aug 24, 2020
michalpristas
michalpristas approved these changes Aug 24, 2020
View reviewed changes
Copy link
Contributor

@michalpristas michalpristas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backport looks ok

@jsoriano
Copy link
Member Author

/package

@elasticmachine
Copy link
Collaborator

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20753 opened]

  • Start Time: 2020-08-24T09:10:40.186+0000

  • Duration: 73 min 37 sec

Test stats 🧪

Test Results
Failed 12
Passed 17980
Skipped 1853
Total 19845

Test errors

Expand to view the tests failures

  • Name: Build and Test / Auditbeat oss Linux / test_dashboards – auditbeat.tests.system.test_base.Test

    • Age: 1
    • Duration: 0.35
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_dev_tool_export_dashboard_by_id – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.259
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_dev_tool_export_dashboard_by_id_from_space – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.188
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_dev_tool_export_dashboard_from_yml – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.199
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_export_dashboard_cmd_export_dashboard_by_id – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.19
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_export_dashboard_cmd_export_dashboard_by_id_and_decoding – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.152
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_export_dashboard_cmd_export_dashboard_from_yml – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.145
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_load_dashboard – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.136
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_load_dashboard_into_space – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 1.029
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_load_only_index_patterns – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.229
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Libbeat / Libbeat oss / test_load_without_dashboard – libbeat.tests.system.test_dashboard.Test

    • Age: 1
    • Duration: 0.262
    • Error Details: AssertionError: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Metricbeat OSS Python Integration tests / test_module_processors – metricbeat.module.redis.test_redis.Test_2

    • Age: 1
    • Duration: 1.034
    • Error Details: AssertionError: Regex matched: '\tWARN\t' matches '\tERROR\t|\tWARN\t' in '2020-08-24T10:06:49.490Z\tINFO\tinstance/beat.go:640\tHome path: [/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors] Config path: [/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors] Data path: [/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors/data] Logs path: [/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors/logs]\n2020-08-24T10:06:49.490Z\tDEBUG\t[beat]\tinstance/beat.go:692\tBeat metadata path: /go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors/data/meta.json\n2020-08-24T10:06:49.494Z\tINFO\tinstance/beat.go:648\tBeat ID: 0155a5d4-52ab-4d8f-8ed1-a63ef0a23827\n2020-08-24T10:06:49.495Z\tDEBUG\t[seccomp]\tseccomp/seccomp.go:117\tLoading syscall filter\t{"seccomp_filter": {"no_new_privs":true,"flag":"tsync","policy":{"default_action":"errno","syscalls":[{"names":["accept","accept4","access","arch_prctl","bind","brk","chmod","chown","clock_gettime","clone","close","connect","dup","dup2","epoll_create","epoll_create1","epoll_ctl","epoll_pwait","epoll_wait","exit","exit_group","fchdir","fchmod","fchmodat","fchown","fchownat","fcntl","fdatasync","flock","fstat","fstatfs","fsync","ftruncate","futex","getcwd","getdents","getdents64","geteuid","getgid","getpeername","getpid","getppid","getrandom","getrlimit","getrusage","getsockname","getsockopt","gettid","gettimeofday","getuid","inotify_add_watch","inotify_init1","inotify_rm_watch","ioctl","kill","listen","lseek","lstat","madvise","mincore","mkdirat","mmap","mprotect","munmap","nanosleep","newfstatat","open","openat","pipe","pipe2","poll","ppoll","pread64","pselect6","pwrite64","read","readlink","readlinkat","recvfrom","recvmmsg","recvmsg","rename","renameat","rt_sigaction","rt_sigprocmask","rt_sigreturn","sched_getaffinity","sched_yield","sendfile","sendmmsg","sendmsg","sendto","set_robust_list","setitimer","setsockopt","shutdown","sigaltstack","socket","splice","stat","statfs","sysinfo","tgkill","time","tkill","uname","unlink","unlinkat","wait4","waitid","write","writev"],"action":"allow"}]}}}\n2020-08-24T10:06:49.496Z\tWARN\t[seccomp]\tseccomp/seccomp.go:119\tSyscall filter could not be installederrorfailed loading seccomp filter: permission deniedseccomp_filter{true tsync {errno [{[accept accept4 access arch_prctl bind brk chmod chown clock_gettime clone close connect dup dup2 epoll_create epoll_create1 epoll_ctl epoll_pwait epoll_wait exit exit_group fchdir fchmod fchmodat fchown fchownat fcntl fdatasync flock fstat fstatfs fsync ftruncate futex getcwd getdents getdents64 geteuid getgid getpeername getpid getppid getrandom getrlimit getrusage getsockname getsockopt gettid gettimeofday getuid inotify_add_watch inotify_init1 inotify_rm_watch ioctl kill listen lseek lstat madvise mincore mkdirat mmap mprotect munmap nanosleep newfstatat open openat pipe pipe2 poll ppoll pread64 pselect6 pwrite64 read readlink readlinkat recvfrom recvmmsg recvmsg rename renameat rt_sigaction rt_sigprocmask rt_sigreturn sched_getaffinity sched_yield sendfile sendmmsg sendmsg sendto set_robust_list setitimer setsockopt shutdown sigaltstack socket splice stat statfs sysinfo tgkill time tkill uname unlink unlinkat wait4 waitid write writev] allow }] }}\n2020-08-24T10:06:49.496Z\tINFO\t[beat]\tinstance/beat.go:976\tBeat info\t{"system_info": {"beat": {"path": {"config": "/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors", "data": "/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors/data", "home": "/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors", "logs": "/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors/logs"}, "type": "metricbeat", "uuid": "0155a5d4-52ab-4d8f-8ed1-a63ef0a23827"}}}\n2020-08-24T10:06:49.496Z\tINFO\t[beat]\tinstance/beat.go:985\tBuild info\t{"system_info": {"build": {"commit": "unknown", "libbeat": "7.10.0", "time": "1754-08-30T22:43:41.128Z", "version": "7.10.0"}}}\n2020-08-24T10:06:49.496Z\tINFO\t[beat]\tinstance/beat.go:988\tGo runtime info\t{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.14.7"}}}\n2020-08-24T10:06:49.498Z\tINFO\t[beat]\tinstance/beat.go:992\tHost info\t{"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-08-24T09:33:06Z","containerized":true,"name":"beats-ci-immutable-ubuntu-1604-1598261562854086863","ip":["127.0.0.1/8","::1/128","10.224.0.252/32","fe80::4001:aff:fee0:fc/64","172.17.0.1/16","fe80::42:12ff:fe1e:516f/64","172.18.0.1/16","172.19.0.1/16","fe80::42:a3ff:fe1e:5abd/64","192.168.112.1/20","fe80::42:8aff:fee2:1bef/64","fe80::90c8:50ff:fee9:9b4d/64"],"kernel_version":"4.15.0-1080-gcp","mac":["42:01:0a:e0:00:fc","02:42:12:1e:51:6f","02:42:c8:60:cf:65","02:42:a3:1e:5a:bd","02:42:8a:e2:1b:ef","92:c8:50:e9:9b:4d"],"os":{"family":"debian","platform":"debian","name":"Debian GNU/Linux","version":"10 (buster)","major":10,"minor":0,"patch":0,"codename":"buster"},"timezone":"UTC","timezone_offset_sec":0,"id":"a8eb6cac33e701ae867269db5ce80e7f"}}}\n2020-08-24T10:06:49.499Z\tINFO\t[beat]\tinstance/beat.go:1021\tProcess info\t{"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/go/src/github.com/elastic/beats/metricbeat", "exe": "/go/src/github.com/elastic/beats/metricbeat/metricbeat.test", "name": "metricbeat.test", "pid": 17125, "ppid": 14919, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2020-08-24T10:06:49.160Z"}}}\n2020-08-24T10:06:49.499Z\tINFO\tinstance/beat.go:299\tSetup Beat: metricbeat; Version: 7.10.0\n2020-08-24T10:06:49.499Z\tDEBUG\t[beat]\tinstance/beat.go:325\tInitializing output plugins\n2020-08-24T10:06:49.499Z\tDEBUG\t[rotator]\tfile/rotator.go:176\tInitialized file rotator\t{"rotator": {"filename": "/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors/output/metricbeat", "max_size_bytes": 1024000, "max_backups": 7, "permissions": "-rw-------", "interval": 0}}\n2020-08-24T10:06:49.499Z\tINFO\t[file]\tfileout/file.go:101\tInitialized file output. path=/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors/output/metricbeat max_size_bytes=1024000 max_backups=7 permissions=-rw-------\n2020-08-24T10:06:49.499Z\tDEBUG\t[publisher]\tpipeline/consumer.go:148\tstart pipeline event consumer\n2020-08-24T10:06:49.500Z\tINFO\t[publisher]\tpipeline/module.go:113\tBeat name: beats-ci-immutable-ubuntu-1604-1598261562854086863\n2020-08-24T10:06:49.500Z\tDEBUG\t[registry.lightmodules]\tmb/lightmodules.go:262\tLight modules directory '%!d(string=/go/src/github.com/elastic/beats/metricbeat/build/system-tests/run/test_redis.Test_2.test_module_processors/module)' doesn't exist\n2020-08-24T10:06:49.500Z\tDEBUG\t[modules]\tbeater/metricbeat.go:151\tAvailable modules and metricsets: Register [ModuleFactory:[beat, docker, elasticsearch, kibana, linux, logstash, mongodb, mysql, postgresql, system, uwsgi], MetricSetFactory:[aerospike/namespace, apache/status, beat/state, beat/stats, ceph/cluster_disk, ceph/cluster_health, ceph/cluster_status, ceph/mgr_cluster_disk, ceph/mgr_cluster_health, ceph/mgr_osd_perf, ceph/mgr_osd_pool_stats, ceph/mgr_osd_tree, ceph/mgr_pool_disk, ceph/monitor_health, ceph/osd_df, ceph/osd_tree, ceph/pool_disk, consul/agent, couchbase/bucket, couchbase/cluster, couchbase/node, couchdb/server, docker/container, docker/cpu, docker/diskio, docker/event, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, dropwizard/collector, elasticsearch/ccr, elasticsearch/cluster_stats, elasticsearch/enrich, elasticsearch/index, elasticsearch/index_recovery, elasticsearch/index_summary, elasticsearch/ml_job, elasticsearch/node, elasticsearch/node_stats, elasticsearch/pending_tasks, elasticsearch/shard, envoyproxy/server, etcd/leader, etcd/metrics, etcd/self, etcd/store, golang/expvar, golang/heap, graphite/server, haproxy/info, haproxy/stat, http/json, http/server, jolokia/jmx, kafka/consumergroup, kafka/partition, kibana/stats, kibana/status, kubernetes/apiserver, kubernetes/container, kubernetes/controllermanager, kubernetes/event, kubernetes/node, kubernetes/pod, kubernetes/proxy, kubernetes/scheduler, kubernetes/state_container, kubernetes/state_cronjob, kubernetes/state_deployment, kubernetes/state_node, kubernetes/state_persistentvolume, kubernetes/state_persistentvolumeclaim, kubernetes/state_pod, kubernetes/state_replicaset, kubernetes/state_resourcequota, kubernetes/state_service, kubernetes/state_statefulset, kubernetes/state_storageclass, kubernetes/system, kubernetes/volume, kvm/dommemstat, kvm/status, linux/conntrack, linux/ksm, linux/pageinfo, logstash/node, logstash/node_stats, memcached/stats, mongodb/collstats, mongodb/dbstats, mongodb/metrics, mongodb/replstatus, mongodb/status, munin/node, mysql/galera_status, mysql/query, mysql/status, nats/connections, nats/routes, nats/stats, nats/subscriptions, nginx/stubstatus, php_fpm/pool, php_fpm/process, postgresql/activity, postgresql/bgwriter, postgresql/database, postgresql/statement, prometheus/collector, prometheus/query, prometheus/remote_write, rabbitmq/connection, rabbitmq/exchange, rabbitmq/node, rabbitmq/queue, redis/info, redis/key, redis/keyspace, system/core, system/cpu, system/diskio, system/entropy, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/network_summary, system/process, system/process_summary, system/raid, system/service, system/socket, system/socket_summary, system/uptime, system/users, traefik/health, uwsgi/status, vsphere/datastore, vsphere/host, vsphere/virtualmachine, zookeeper/connection, zookeeper/mntr, zookeeper/server], LightModules:[LightModules:[]]]\n2020-08-24T10:06:49.500Z\tDEBUG\t[processors]\tprocessors/processor.go:101\tGenerated new processors: include_fields=beat, metricset, service, event, redis.info.clients, redis.info.cpu, type\n2020-08-24T10:06:49.500Z\tINFO\t[monitoring]\tlog/log.go:118\tStarting metrics logging every 30s\n2020-08-24T10:06:49.500Z\tINFO\tinstance/beat.go:450\tmetricbeat start running.\n2020-08-24T10:06:49.500Z\tDEBUG\t[module]\tmodule/wrapper.go:127\tStarting Wrapper[name=redis, len(metricSetWrappers)=1]\n2020-08-24T10:06:49.500Z\tDEBUG\t[module]\tmodule/wrapper.go:189\tStarting metricSetWrapper[module=redis, name=info, host=192.168.112.2:6379]\n2020-08-24T10:06:49.502Z\tDEBUG\t[redis.info]\tinfo/info.go:90\tRedis INFO from 192.168.112.2:6379: map[active_defrag_hits:0 active_defrag_key_hits:0 active_defrag_key_misses:0 active_defrag_misses:0 active_defrag_running:0 allocator_active:1245184 allocator_allocated:1047992 allocator_frag_bytes:197192 allocator_frag_ratio:1.19 allocator_resident:3825664 allocator_rss_bytes:2580480 allocator_rss_ratio:3.07 aof_current_rewrite_time_sec:-1 aof_enabled:0 aof_last_bgrewrite_status:ok aof_last_cow_size:0 aof_last_rewrite_time_sec:-1 aof_last_write_status:ok aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 arch_bits:64 atomicvar_api:atomic-builtin blocked_clients:0 client_biggest_input_buf:2 client_longest_output_list:0 client_recent_max_input_buffer:2 client_recent_max_output_buffer:0 cluster_enabled:0 config_file: configured_hz:10 connected_clients:1 connected_slaves:0 db0:keys=1,expires=0,avg_ttl=0 evicted_keys:0 executable:/data/redis-server expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 gcc_version:8.3.0 hz:10 instantaneous_input_kbps:0.18 instantaneous_ops_per_sec:6 instantaneous_output_kbps:0.07 keyspace_hits:3 keyspace_misses:0 latest_fork_usec:0 lazyfree_pending_objects:0 loading:0 lru_clock:4428089 master_repl_offset:0 master_replid:26ae6596b775dc77ff42153943929fa75eaea6f7 master_replid2:0000000000000000000000000000000000000000 maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction mem_allocator:jemalloc-5.1.0 mem_aof_buffer:0 mem_clients_normal:49694 mem_clients_slaves:0 mem_fragmentation_bytes:1969360 mem_fragmentation_ratio:3.49 mem_not_counted_for_evict:0 mem_replication_backlog:0 migrate_cached_sockets:0 multiplexing_api:epoll number_of_cached_scripts:0 os:Linux 4.15.0-1080-gcp x86_64 process_id:1 pubsub_channels:0 pubsub_patterns:0 rdb_bgsave_in_progress:0 rdb_changes_since_last_save:7 rdb_current_bgsave_time_sec:-1 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:-1 rdb_last_cow_size:0 rdb_last_save_time:1598263608 redis_build_id:4d072dc1c62d5672 redis_git_dirty:0 redis_git_sha1:00000000 redis_mode:standalone redis_version:5.0.5 rejected_connections:0 repl_backlog_active:0 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0 repl_backlog_size:1048576 role:master rss_overhead_bytes:-1064960 rss_overhead_ratio:0.72 run_id:399bf0e1955f5fabbd4786ab7a1148afb20f5e6c second_repl_offset:-1 slave_expires_tracked_keys:0 slowlog_len:0 sync_full:0 sync_partial_err:0 sync_partial_ok:0 tcp_port:6379 total_commands_processed:12 total_connections_received:9 total_net_input_bytes:385 total_net_output_bytes:3352 total_system_memory:27388301312 total_system_memory_human:25.51G uptime_in_days:0 uptime_in_seconds:4 used_cpu_sys:0.008627 used_cpu_sys_children:0.000000 used_cpu_user:0.022878 used_cpu_user_children:0.001826 used_memory:854264 used_memory_dataset:13258 used_memory_dataset_perc:21.04% used_memory_human:834.24K used_memory_lua:37888 used_memory_lua_human:37.00K used_memory_overhead:841006 used_memory_peak:874224 used_memory_peak_human:853.73K used_memory_peak_perc:97.72% used_memory_rss:2760704 used_memory_rss_human:2.63M used_memory_scripts:0 used_memory_scripts_human:0B used_memory_startup:791240]\n2020-08-24T10:06:49.502Z\tDEBUG\t[processors]\tprocessing/processors.go:187\tPublish event: {\n "@timestamp": "2020-08-24T10:06:49.500Z",\n "@metadata": {\n "beat": "metricbeat",\n "type": "_doc",\n "version": "7.10.0"\n },\n "host": {\n "name": "beats-ci-immutable-ubuntu-1604-1598261562854086863"\n },\n "agent": {\n "name": "beats-ci-immutable-ubuntu-1604-1598261562854086863",\n "type": "metricbeat",\n "version": "7.10.0",\n "hostname": "beats-ci-immutable-ubuntu-1604-1598261562854086863",\n "ephemeral_id": "00fa12e2-b01b-4a43-94fd-3b01fc8fc639",\n "id": "0155a5d4-52ab-4d8f-8ed1-a63ef0a23827"\n },\n "ecs": {\n "version": "1.5.0"\n },\n "metricset": {\n "name": "info",\n "period": 5000\n },\n "service": {\n "version": "5.0.5",\n "address": "192.168.112.2:6379",\n "type": "redis"\n },\n "event": {\n "dataset": "redis.info",\n "module": "redis",\n "duration": 1727983\n },\n "redis": {\n "info": {\n "clients": {\n "connected": 1,\n "longest_output_list": 0,\n "biggest_input_buf": 2,\n "max_output_buffer": 0,\n "max_input_buffer": 2,\n "blocked": 0\n },\n "cpu": {\n "used": {\n "sys": 0.008627,\n "user": 0.022878,\n "sys_children": 0,\n "user_children": 0.001826\n }\n }\n }\n }\n}\n2020-08-24T10:06:49.603Z\tDEBUG\t[publisher]\tmemqueue/ackloop.go:160\tackloop: receive ack [0: 0, 1]\n2020-08-24T10:06:49.603Z\tDEBUG\t[publisher]\tmemqueue/ackloop.go:128\tackloop: return ack to broker loop:1\n2020-08-24T10:06:49.603Z\tDEBUG\t[publisher]\tmemqueue/ackloop.go:131\tackloop: done send ack\n2020-08-24T10:06:49.621Z\tDEBUG\t[service]\tservice/service.go:54\tReceived sigterm/sigint, stopping\n2020-08-24T10:06:49.621Z\tDEBUG\t[publisher]\tpipeline/client.go:157\tclient: closing acker\n2020-08-24T10:06:49.621Z\tDEBUG\t[publisher]\tpipeline/client.go:162\tclient: done closing acker\n2020-08-24T10:06:49.621Z\tDEBUG\t[publisher]\tpipeline/client.go:164\tclient: unlink from queue\n2020-08-24T10:06:49.622Z\tDEBUG\t[publisher]\tpipeline/client.go:177\tclient: cancelled 0 events\n2020-08-24T10:06:49.622Z\tDEBUG\t[publisher]\tpipeline/client.go:166\tclient: done unlink\n2020-08-24T10:06:49.622Z\tDEBUG\t[module]\tmodule/wrapper.go:214\tStopped metricSetWrapper[module=redis, name=info, host=192.168.112.2:6379]\n2020-08-24T10:06:49.622Z\tDEBUG\t[module]\tmodule/wrapper.go:155\tStopped Wrapper[name=redis, len(metricSetWrappers)=1]\n2020-08-24T10:06:49.624Z\tINFO\t[monitoring]\tlog/log.go:153\tTotal non-zero metrics\t{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":54}},"total":{"ticks":120,"time":{"ms":133},"value":120},"user":{"ticks":70,"time":{"ms":79}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"00fa12e2-b01b-4a43-94fd-3b01fc8fc639","uptime":{"ms":188}},"memstats":{"gc_next":9149232,"memory_alloc":4902136,"memory_total":17058584,"rss":56721408},"runtime":{"goroutines":14}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":1,"batches":1,"total":1},"type":"file","write":{"bytes":898}},"pipeline":{"clients":0,"events":{"active":0,"published":1,"total":1},"queue":{"acked":1}}},"system":{"cpu":{"cores":4},"load":{"1":0.39,"15":1.82,"5":1.42,"norm":{"1":0.0975,"15":0.455,"5":0.355}}}}}}\n2020-08-24T10:06:49.624Z\tINFO\t[monitoring]\tlog/log.go:154\tUptime: 188.865752ms\n2020-08-24T10:06:49.624Z\tINFO\t[monitoring]\tlog/log.go:131\tStopping metrics logging.\n2020-08-24T10:06:49.624Z\tINFO\tinstance/beat.go:456\tmetricbeat stopped.\nPASS\ncoverage: 7.4% of statements in ./...\n'

Steps errors

Expand to view the steps failures

  • Name: Mage build test

    • Description: mage build test

    • Duration: 11 min 21 sec

    • Start Time: 2020-08-24T09:34:32.181+0000

    • log

  • Name: Mage build test

    • Description: mage build test

    • Duration: 30 min 1 sec

    • Start Time: 2020-08-24T09:34:34.503+0000

    • log

  • Name: Mage pythonIntegTest

    • Description: mage pythonIntegTest

    • Duration: 34 min 56 sec

    • Start Time: 2020-08-24T09:34:19.069+0000

    • log

Log output

Expand to view the last 100 lines of log output 

[2020-08-24T10:22:23.347Z] [Google Cloud Storage Plugin] Downloading: Beats/beats/PR-20753-1/source/source.tgz to local path: /var/lib/jenkins/workspace/Beats_beats_PR-20753/source.tgz
[2020-08-24T10:22:31.991Z] + tar --version
[2020-08-24T10:22:32.308Z] + tar -xpf source.tgz
[2020-08-24T10:22:42.675Z] + rm source.tgz
[2020-08-24T10:22:42.700Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats
[2020-08-24T10:22:42.750Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Lint
[2020-08-24T10:22:42.912Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Winlogbeat-oss
[2020-08-24T10:22:43.072Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-08-24T10:22:43.235Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-08-24T10:22:43.409Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/dockerlogbeat
[2020-08-24T10:22:43.573Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-08-24T10:22:43.765Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Journalbeat
[2020-08-24T10:22:44.088Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-08-24T10:22:44.404Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-08-24T10:22:44.593Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-08-24T10:22:44.759Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Packetbeat-Linux
[2020-08-24T10:22:44.934Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-08-24T10:22:45.097Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-08-24T10:22:45.255Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-08-24T10:22:45.414Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-08-24T10:22:45.572Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-08-24T10:22:45.733Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-08-24T10:22:45.896Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-08-24T10:22:46.055Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-08-24T10:22:46.213Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Heartbeat-oss
[2020-08-24T10:22:46.373Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-08-24T10:22:46.533Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-08-24T10:22:46.695Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Heartbeat-Windows
[2020-08-24T10:22:46.858Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-08-24T10:22:47.024Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-08-24T10:22:47.188Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-08-24T10:22:47.417Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Filebeat-Windows
[2020-08-24T10:22:47.577Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Packetbeat-Windows
[2020-08-24T10:22:47.740Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Libbeat-x-pack
[2020-08-24T10:22:47.898Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-08-24T10:22:48.056Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Functionbeat-Windows
[2020-08-24T10:22:48.219Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-08-24T10:22:48.377Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-08-24T10:22:48.534Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Packetbeat-Mac-OS-X
[2020-08-24T10:22:48.699Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-08-24T10:22:48.853Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-08-24T10:22:49.009Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-08-24T10:22:49.167Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-Windows
[2020-08-24T10:22:49.325Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-08-24T10:22:49.493Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-08-24T10:22:49.656Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Filebeat-oss
[2020-08-24T10:22:49.814Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Libbeat-oss
[2020-08-24T10:22:49.971Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Filebeat-x-pack
[2020-08-24T10:22:50.133Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests
[2020-08-24T10:22:50.295Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-OSS-Python-Integration-tests
[2020-08-24T10:22:50.456Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-08-24T10:22:50.978Z] + cat
[2020-08-24T10:22:50.978Z] + /usr/local/bin/runbld ./runbld-script --job-name elastic+beats+pull-request
[2020-08-24T10:22:50.978Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-08-24T10:22:57.571Z] runbld>>> runbld started
[2020-08-24T10:22:57.571Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-08-24T10:22:59.487Z] runbld>>> The following profiles matched the job 'elastic+beats+pull-request' in order of occurrence in the config (last value wins).
[2020-08-24T10:22:59.487Z] runbld>>> Matches in the system config:
[2020-08-24T10:22:59.487Z] runbld>>> - Matched ^elastic\+beats
[2020-08-24T10:22:59.487Z] runbld>>> - Matched ^elastic\+beats\+pull-request
[2020-08-24T10:23:00.443Z] runbld>>> Debug logging enabled.
[2020-08-24T10:23:00.443Z] runbld>>> Storing result
[2020-08-24T10:23:00.705Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-08-24T10:23:00.705Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200824102300-7837E2D0
[2020-08-24T10:23:00.705Z] runbld>>> Adding system facts.
[2020-08-24T10:23:01.655Z] runbld>>> Adding vcs info for the latest commit:  d7acf80256fbe60c85edfe8fc5579d7f897635c8
[2020-08-24T10:23:01.655Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-08-24T10:23:01.655Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-08-24T10:23:01.916Z] Processing JUnit reports with runbld...
[2020-08-24T10:23:01.916Z] + echo 'Processing JUnit reports with runbld...'
[2020-08-24T10:23:02.177Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-08-24T10:23:02.177Z] runbld>>> DURATION: 24ms
[2020-08-24T10:23:02.177Z] runbld>>> STDOUT: 40 bytes
[2020-08-24T10:23:02.177Z] runbld>>> STDERR: 49 bytes
[2020-08-24T10:23:02.177Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-08-24T10:23:02.177Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats_PR-20753
[2020-08-24T10:23:03.120Z] runbld>>> Storing build metadata: 
[2020-08-24T10:23:03.121Z] runbld>>> Adding test report.
[2020-08-24T10:23:03.121Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats
[2020-08-24T10:23:03.693Z] runbld>>> Found 137 test output files
[2020-08-24T10:23:06.247Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-08-24T10:23:06.247Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-08-24T10:23:06.247Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-08-24T10:23:06.247Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-08-24T10:23:06.247Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-08-24T10:23:06.247Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-08-24T10:23:06.247Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20753/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-08-24T10:23:07.195Z] runbld>>> Test output logs contained: Errors: 0 Failures: 12 Tests: 19694 Skipped: 1575
[2020-08-24T10:23:07.457Z] runbld>>> Storing result
[2020-08-24T10:23:07.457Z] runbld>>> FAILURES: 12
[2020-08-24T10:23:10.011Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-08-24T10:23:10.011Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200824102300-7837E2D0
[2020-08-24T10:23:10.011Z] runbld>>> Email notification disabled by environment variable.
[2020-08-24T10:23:10.011Z] runbld>>> Slack notification disabled by environment variable.
[2020-08-24T10:23:15.688Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-20753
[2020-08-24T10:23:15.862Z] [INFO] getVaultSecret: Getting secrets
[2020-08-24T10:23:15.936Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-08-24T10:23:16.998Z] + chmod 755 generate-build-data.sh
[2020-08-24T10:23:16.998Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20753/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20753/runs/1 FAILURE 4356541
[2020-08-24T10:23:16.998Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20753/runs/1/steps/?limit=10000 -o steps-info.json

@jsoriano
Copy link
Member Author

Failing tests are not related to this change, they are also failing in 7.x branch, merging.

@jsoriano jsoriano merged commit c44fe6d into elastic:7.x Aug 24, 2020
@jsoriano jsoriano deleted the backport_20576_7.x branch August 24, 2020 11:00
@jsoriano
Copy link
Member Author

Failing tests are caused by #20716

@axw axw mentioned this pull request Aug 25, 2020
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michalpristas michalpristas michalpristas approved these changes

Assignees
No one assigned
Labels
backport Team:Integrations Label for the Integrations team Team:Platforms Label for the Integrations - Platforms team [zube]: In Review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jsoriano @elasticmachine @michalpristas