Cherry-pick to 7.12: [PACKAGING] Push docker images with the architecture in the version (#24121) #24159

v1v · 2021-02-22T14:57:34Z

Backports the following commits to 7.12:

[PACKAGING] Push docker images with the architecture in the version ([PACKAGING] Push docker images with the architecture in the version #24121)

…lastic#24121)

botelastic · 2021-02-22T14:57:44Z

This pull request doesn't have a Team:<team> label.

elasticmachine · 2021-02-22T16:34:02Z

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #24159 opened
Start Time: 2021-02-22T14:58:37.063+0000
Duration: 94 min 27 sec
Commit: 54a95e4

Test stats 🧪

Test	Results
Failed	1
Passed	46576
Skipped	4981
Total	51558

Trends 🧪

Test errors

Expand to view the tests failures

`Build&Test / x-pack/filebeat-build / test_fileset_file_177_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['log.offset']": {'new_value': 79352, 'old_value': 78028}, "root['cisco.amp.detection_id']": {'new_value': '6159251516445163617', 'old_value': '6159251516445163618'}}}, full object:    {'log.offset': 79352, 'fileset.name': 'amp', 'tags': ['cisco-amp', 'forwarded'], 'input.type': 'log', '@timestamp': '2021-01-15T10:17:39.000Z', 'file.path': '\\\\?\\C:\\Users\\Administrator\\AppData\\Roaming\\rjtsbks.exe', 'file.name': 'rjtsbks.exe', 'file.hash.sha1': 'e654d39cd13414b5151e8cf0d8f5b166dddd45cb', 'file.hash.sha256': '3372c1edab46837f1e973164fa2d726c5c5e17bcb888828ccd7c4dfcc234a370', 'file.hash.md5': '209a288c68207d57e0ce6e60ebf60729', 'related.hosts': ['Demo_TeslaCrypt'], 'related.ip': ['8.8.8.8', '10.10.10.10'], 'related.hash': ['3372c1edab46837f1e973164fa2d726c5c5e17bcb888828ccd7c4dfcc234a370', '209a288c68207d57e0ce6e60ebf60729', 'e654d39cd13414b5151e8cf0d8f5b166dddd45cb'], 'service.type': 'cisco', 'host.hostname': 'Demo_TeslaCrypt', 'host.os.family': 'windows', 'host.os.platform': 'windows', 'host.name': 'Demo_TeslaCrypt', 'event.severity': 2, 'event.kind': 'alert', 'event.module': 'cisco', 'event.action': 'Threat Detected', 'event.id': 6159251516445164000, 'event.category': ['file', 'malware'], 'event.dataset': 'cisco.amp', 'cisco.amp.detection': 'W32.DFC.MalParent', 'cisco.amp.group_guids': ['test_group_guid'], 'cisco.amp.event_type_id': 1090519054, 'cisco.amp.timestamp_nanoseconds': 988000000, 'cisco.amp.computer.network_addresses': [{'ip': '10.10.10.10', 'mac': '90:61:b5:c9:13:79'}], 'cisco.amp.computer.connector_guid': 'test_connector_guid', 'cisco.amp.computer.active': True, 'cisco.amp.computer.external_ip': '8.8.8.8', 'cisco.amp.file.disposition': 'Malicious', 'cisco.amp.connector_guid': 'test_connector_guid', 'cisco.amp.related.mac': ['90:61:b5:c9:13:79'], 'cisco.amp.detection_id': '6159251516445163617'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['log.offset']": {'new_value': 79352, 'old_value': 78028}, "root['cisco.amp.detection_id']": {'new_value': '6159251516445163617', 'old_value': '6159251516445163618'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_177_cisco>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_177_cisco>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cisco/amp/test/cisco_amp2.ndjson.log'
objects = [{'@timestamp': '2021-01-15T11:59:52.000Z', 'agent': {'ephemeral_id': 'f3c506c6-148e-4e1e-977d-bbb74ff582a1', 'hostnam...}, 'connector_guid': 'test_connector_guid', 'event_type_id': 1107296274, ...}}, 'ecs': {'version': '1.7.0'}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['log.offset']": {'new_value': 79352, 'old_value': 78028}, "root['cisco.amp.detection_id']": {'new_value': '6159251516445163617', 'old_value': '6159251516445163618'}}}, full object: 
E             {'log.offset': 79352, 'fileset.name': 'amp', 'tags': ['cisco-amp', 'forwarded'], 'input.type': 'log', '@timestamp': '2021-01-15T10:17:39.000Z', 'file.path': '\\\\?\\C:\\Users\\Administrator\\AppData\\Roaming\\rjtsbks.exe', 'file.name': 'rjtsbks.exe', 'file.hash.sha1': 'e654d39cd13414b5151e8cf0d8f5b166dddd45cb', 'file.hash.sha256': '3372c1edab46837f1e973164fa2d726c5c5e17bcb888828ccd7c4dfcc234a370', 'file.hash.md5': '209a288c68207d57e0ce6e60ebf60729', 'related.hosts': ['Demo_TeslaCrypt'], 'related.ip': ['8.8.8.8', '10.10.10.10'], 'related.hash': ['3372c1edab46837f1e973164fa2d726c5c5e17bcb888828ccd7c4dfcc234a370', '209a288c68207d57e0ce6e60ebf60729', 'e654d39cd13414b5151e8cf0d8f5b166dddd45cb'], 'service.type': 'cisco', 'host.hostname': 'Demo_TeslaCrypt', 'host.os.family': 'windows', 'host.os.platform': 'windows', 'host.name': 'Demo_TeslaCrypt', 'event.severity': 2, 'event.kind': 'alert', 'event.module': 'cisco', 'event.action': 'Threat Detected', 'event.id': 6159251516445164000, 'event.category': ['file', 'malware'], 'event.dataset': 'cisco.amp', 'cisco.amp.detection': 'W32.DFC.MalParent', 'cisco.amp.group_guids': ['test_group_guid'], 'cisco.amp.event_type_id': 1090519054, 'cisco.amp.timestamp_nanoseconds': 988000000, 'cisco.amp.computer.network_addresses': [{'ip': '10.10.10.10', 'mac': '90:61:b5:c9:13:79'}], 'cisco.amp.computer.connector_guid': 'test_connector_guid', 'cisco.amp.computer.active': True, 'cisco.amp.computer.external_ip': '8.8.8.8', 'cisco.amp.file.disposition': 'Malicious', 'cisco.amp.connector_guid': 'test_connector_guid', 'cisco.amp.related.mac': ['90:61:b5:c9:13:79'], 'cisco.amp.detection_id': '6159251516445163617'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['log.offset']": {'new_value': 79352, 'old_value': 78028}, "root['cisco.amp.detection_id']": {'new_value': '6159251516445163617', 'old_value': '6159251516445163618'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

Steps errors

Expand to view the steps failures

`x-pack/filebeat-build - mage build test`

Took 35 min 4 sec . View more details on here
Description: mage build test

`Print Message`

Took 0 min 0 sec . View more details on here
Description: �[39;49m[INFO] getVaultSecret: Getting secrets�[0m

`Error signal`

Took 0 min 0 sec . View more details on here
Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-02-22T16:30:35.619Z]  Context:           default
[2021-02-22T16:30:35.619Z]  Experimental:      true
[2021-02-22T16:30:35.619Z] 
[2021-02-22T16:30:35.619Z] Server: Docker Engine - Community
[2021-02-22T16:30:35.619Z]  Engine:
[2021-02-22T16:30:35.619Z]   Version:          20.10.3
[2021-02-22T16:30:35.619Z]   API version:      1.41 (minimum version 1.12)
[2021-02-22T16:30:35.619Z]   Go version:       go1.13.15
[2021-02-22T16:30:35.619Z]   Git commit:       46229ca
[2021-02-22T16:30:35.619Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-22T16:30:35.619Z]   OS/Arch:          linux/amd64
[2021-02-22T16:30:35.619Z]   Experimental:     false
[2021-02-22T16:30:35.619Z]  containerd:
[2021-02-22T16:30:35.619Z]   Version:          1.4.3
[2021-02-22T16:30:35.619Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-22T16:30:35.619Z]  runc:
[2021-02-22T16:30:35.619Z]   Version:          1.0.0-rc92
[2021-02-22T16:30:35.619Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-22T16:30:35.619Z]  docker-init:
[2021-02-22T16:30:35.619Z]   Version:          0.19.0
[2021-02-22T16:30:35.619Z]   GitCommit:        de40ad0
[2021-02-22T16:30:35.619Z] Unable to find image 'alpine:3.4' locally
[2021-02-22T16:30:36.562Z] 3.4: Pulling from library/alpine
[2021-02-22T16:30:36.562Z] c1e54eec4b57: Pulling fs layer
[2021-02-22T16:30:36.823Z] c1e54eec4b57: Verifying Checksum
[2021-02-22T16:30:36.823Z] c1e54eec4b57: Download complete
[2021-02-22T16:30:37.083Z] c1e54eec4b57: Pull complete
[2021-02-22T16:30:37.083Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-02-22T16:30:37.083Z] Status: Downloaded newer image for alpine:3.4
[2021-02-22T16:30:39.941Z] + python .ci/scripts/pre_archive_test.py
[2021-02-22T16:30:42.481Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-02-22T16:30:42.558Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24159/src/github.com/elastic/beats/build
[2021-02-22T16:30:43.021Z] + rm -rf ve
[2021-02-22T16:30:43.021Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-02-22T16:30:43.096Z] Recording test results
[2021-02-22T16:30:46.822Z] [Checks API] No suitable checks publisher found.
[2021-02-22T16:30:47.168Z] + tar --version
[2021-02-22T16:30:47.588Z] + tar --exclude=test-build-artifacts-x-pack/filebeat-build.tgz -czf test-build-artifacts-x-pack/filebeat-build.tgz .
[2021-02-22T16:31:44.308Z] + gsutil --version
[2021-02-22T16:31:44.308Z] /var/lib/jenkins/workspace/Beats_beats_PR-24159/src/github.com/elastic/beats/build@tmp/durable-f878f24e/script.sh: 1: /var/lib/jenkins/workspace/Beats_beats_PR-24159/src/github.com/elastic/beats/build@tmp/durable-f878f24e/script.sh: gsutil: not found
[2021-02-22T16:31:44.446Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24159/src/github.com/elastic/beats/build@tmp
[2021-02-22T16:31:44.841Z] + wget --version
[2021-02-22T16:31:45.303Z] + wget -q -O gsutil.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-linux-x86_64.tar.gz
[2021-02-22T16:31:46.069Z] + tar -xpf gsutil.tar.gz --strip-components=1
[2021-02-22T16:31:49.715Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-02-22T16:31:50.171Z] + gcloud auth activate-service-account --key-file ****
[2021-02-22T16:31:52.085Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-02-22T16:31:52.562Z] + gsutil -m -q cp -a public-read test-build-artifacts-x-pack/filebeat-build.tgz gs://beats-ci-temp/Beats/beats/PR-24159-1
[2021-02-22T16:31:57.755Z] + python .ci/scripts/search_system_tests.py
[2021-02-22T16:31:57.813Z] [INFO] system-tests='build/x-pack/filebeat/build/system-tests'. If no empty then let's create a tarball
[2021-02-22T16:31:58.175Z] + tar --version
[2021-02-22T16:31:58.576Z] + tar --exclude=x-pack-filebeat--system-tests-linux.tgz -czf x-pack-filebeat--system-tests-linux.tgz build/x-pack/filebeat/build/system-tests
[2021-02-22T16:32:31.546Z] + gsutil --version
[2021-02-22T16:32:31.546Z] /var/lib/jenkins/workspace/Beats_beats_PR-24159/src/github.com/elastic/beats@tmp/durable-3535b077/script.sh: 1: /var/lib/jenkins/workspace/Beats_beats_PR-24159/src/github.com/elastic/beats@tmp/durable-3535b077/script.sh: gsutil: not found
[2021-02-22T16:32:31.705Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24159/src/github.com/elastic/beats@tmp
[2021-02-22T16:32:32.124Z] + wget --version
[2021-02-22T16:32:32.633Z] + wget -q -O gsutil.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-linux-x86_64.tar.gz
[2021-02-22T16:32:33.438Z] + tar -xpf gsutil.tar.gz --strip-components=1
[2021-02-22T16:32:36.952Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-02-22T16:32:37.434Z] + gcloud auth activate-service-account --key-file ****
[2021-02-22T16:32:39.361Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-02-22T16:32:40.122Z] + gsutil -m -q cp -a public-read x-pack-filebeat--system-tests-linux.tgz gs://beats-ci-temp/Beats/beats/PR-24159-1
[2021-02-22T16:32:45.181Z] Client: Docker Engine - Community
[2021-02-22T16:32:45.181Z]  Version:           20.10.3
[2021-02-22T16:32:45.181Z]  API version:       1.41
[2021-02-22T16:32:45.181Z]  Go version:        go1.13.15
[2021-02-22T16:32:45.181Z]  Git commit:        48d30b5
[2021-02-22T16:32:45.181Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-22T16:32:45.181Z]  OS/Arch:           linux/amd64
[2021-02-22T16:32:45.181Z]  Context:           default
[2021-02-22T16:32:45.181Z]  Experimental:      true
[2021-02-22T16:32:45.181Z] 
[2021-02-22T16:32:45.181Z] Server: Docker Engine - Community
[2021-02-22T16:32:45.181Z]  Engine:
[2021-02-22T16:32:45.181Z]   Version:          20.10.3
[2021-02-22T16:32:45.181Z]   API version:      1.41 (minimum version 1.12)
[2021-02-22T16:32:45.181Z]   Go version:       go1.13.15
[2021-02-22T16:32:45.181Z]   Git commit:       46229ca
[2021-02-22T16:32:45.181Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-22T16:32:45.181Z]   OS/Arch:          linux/amd64
[2021-02-22T16:32:45.181Z]   Experimental:     false
[2021-02-22T16:32:45.181Z]  containerd:
[2021-02-22T16:32:45.181Z]   Version:          1.4.3
[2021-02-22T16:32:45.181Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-22T16:32:45.181Z]  runc:
[2021-02-22T16:32:45.181Z]   Version:          1.0.0-rc92
[2021-02-22T16:32:45.181Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-22T16:32:45.181Z]  docker-init:
[2021-02-22T16:32:45.181Z]   Version:          0.19.0
[2021-02-22T16:32:45.181Z]   GitCommit:        de40ad0
[2021-02-22T16:32:55.370Z] Failed in branch x-pack/filebeat-build
[2021-02-22T16:33:01.220Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-02-22T16:33:01.556Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24159/src/github.com/elastic/beats
[2021-02-22T16:33:02.628Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-24159
[2021-02-22T16:33:02.773Z] [INFO] getVaultSecret: Getting secrets
[2021-02-22T16:33:02.914Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-02-22T16:33:04.162Z] + chmod 755 generate-build-data.sh
[2021-02-22T16:33:04.162Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24159/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24159/runs/1 FAILURE 5666795
[2021-02-22T16:33:04.412Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24159/runs/1/steps/?limit=10000 -o steps-info.json
[2021-02-22T16:33:12.783Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24159/runs/1/tests/?status=FAILED -o tests-errors.json

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	1
Passed	46576
Skipped	4981
Total	51558

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Build&Test / x-pack/filebeat-build / test_fileset_file_177_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

[PACKAGING] Push docker images with the architecture in the version (e…

54a95e4

…lastic#24121)

v1v self-assigned this Feb 22, 2021

v1v added the automation label Feb 22, 2021

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 22, 2021

v1v added backport ci Team:Automation Label for the Observability productivity team labels Feb 22, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 22, 2021

v1v requested a review from a team February 22, 2021 18:00

cachedout approved these changes Feb 22, 2021

View reviewed changes

mdelapenya approved these changes Feb 22, 2021

View reviewed changes

v1v merged commit 505e315 into elastic:7.12 Feb 23, 2021

v1v deleted the backport/7.12/pr-24121 branch February 23, 2021 09:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cherry-pick to 7.12: [PACKAGING] Push docker images with the architecture in the version (#24121) #24159

Cherry-pick to 7.12: [PACKAGING] Push docker images with the architecture in the version (#24121) #24159

v1v commented Feb 22, 2021

botelastic bot commented Feb 22, 2021

elasticmachine commented Feb 22, 2021

Build stats

Test stats 🧪

Trends 🧪

`Build&Test / x-pack/filebeat-build / test_fileset_file_177_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`x-pack/filebeat-build - mage build test`

`Print Message`

`Error signal`

Test stats 🧪

Genuine test errors

Cherry-pick to 7.12: [PACKAGING] Push docker images with the architecture in the version (#24121) #24159

Cherry-pick to 7.12: [PACKAGING] Push docker images with the architecture in the version (#24121) #24159

Conversation

v1v commented Feb 22, 2021

botelastic bot commented Feb 22, 2021

elasticmachine commented Feb 22, 2021

💔 Tests Failed

Build stats

Test stats 🧪

Trends 🧪

Test errors

Build&Test / x-pack/filebeat-build / test_fileset_file_177_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Steps errors

x-pack/filebeat-build - mage build test

Print Message

Error signal

Log output

🐛 Flaky test report

Test stats 🧪

Genuine test errors

`Build&Test / x-pack/filebeat-build / test_fileset_file_177_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`x-pack/filebeat-build - mage build test`

`Print Message`

`Error signal`