Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PACKAGING] Push docker images with the architecture in the version #24121

Merged
merged 9 commits into from
Feb 22, 2021

Conversation

v1v
Copy link
Member

@v1v v1v commented Feb 19, 2021

What does this PR do?

Support multiplatform docker images:

  • with the -amd64 in the versioning
  • with the -arm64 in the versioning
  • Default docker image for amd64 arch only. <--- This should eventually be removed once the E2E supports the arch in the versioning.

Enable the E2E testing

Support the above implementation as part of the existing Packaging pipeline and the packaging stage in the pipeline 2.0

Why is it important?

Avoid the override of docker images that are supporting AMD and ARM

@v1v v1v added the skip-ci Skip the build in the CI but linting label Feb 19, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 19, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Feb 19, 2021

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #24121 event

  • Start Time: 2021-02-19T14:07:16.299+0000

  • Duration: 48 min 38 sec

  • Commit: 446c7e1

Test stats 🧪

Test Results
Failed 0
Passed 46694
Skipped 4922
Total 51616

Trends 🧪

Image of Build Times

Image of Tests

Steps errors 3

Expand to view the steps failures

x-pack/metricbeat-windows-7-32-windows-7-32-bit - mage build unitTest
  • Took 4 min 23 sec . View more details on here
  • Description: mage build unitTest
authenticate
  • Took 0 min 1 sec . View more details on here
  • Description: gcloud auth activate-service-account --key-file %FILE_CREDENTIAL%
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-02-19T14:53:51.776Z] 
[2021-02-19T14:53:51.776Z] Server: Docker Engine - Community
[2021-02-19T14:53:51.776Z]  Engine:
[2021-02-19T14:53:51.776Z]   Version:          20.10.3
[2021-02-19T14:53:51.776Z]   API version:      1.41 (minimum version 1.12)
[2021-02-19T14:53:51.776Z]   Go version:       go1.13.15
[2021-02-19T14:53:51.776Z]   Git commit:       46229ca
[2021-02-19T14:53:51.776Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-19T14:53:51.776Z]   OS/Arch:          linux/amd64
[2021-02-19T14:53:51.776Z]   Experimental:     false
[2021-02-19T14:53:51.776Z]  containerd:
[2021-02-19T14:53:51.776Z]   Version:          1.4.3
[2021-02-19T14:53:51.776Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-19T14:53:51.776Z]  runc:
[2021-02-19T14:53:51.776Z]   Version:          1.0.0-rc92
[2021-02-19T14:53:51.776Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-19T14:53:51.776Z]  docker-init:
[2021-02-19T14:53:51.776Z]   Version:          0.19.0
[2021-02-19T14:53:51.776Z]   GitCommit:        de40ad0
[2021-02-19T14:53:51.776Z] Unable to find image 'alpine:3.4' locally
[2021-02-19T14:53:51.850Z] Client: Docker Engine - Community
[2021-02-19T14:53:51.850Z]  Version:           20.10.3
[2021-02-19T14:53:51.850Z]  API version:       1.41
[2021-02-19T14:53:51.850Z]  Go version:        go1.13.15
[2021-02-19T14:53:51.850Z]  Git commit:        48d30b5
[2021-02-19T14:53:51.850Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-19T14:53:51.850Z]  OS/Arch:           linux/amd64
[2021-02-19T14:53:51.850Z]  Context:           default
[2021-02-19T14:53:51.850Z]  Experimental:      true
[2021-02-19T14:53:51.850Z] 
[2021-02-19T14:53:51.850Z] Server: Docker Engine - Community
[2021-02-19T14:53:51.850Z]  Engine:
[2021-02-19T14:53:51.850Z]   Version:          20.10.3
[2021-02-19T14:53:51.850Z]   API version:      1.41 (minimum version 1.12)
[2021-02-19T14:53:51.850Z]   Go version:       go1.13.15
[2021-02-19T14:53:51.850Z]   Git commit:       46229ca
[2021-02-19T14:53:51.850Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-19T14:53:51.850Z]   OS/Arch:          linux/amd64
[2021-02-19T14:53:51.850Z]   Experimental:     false
[2021-02-19T14:53:51.850Z]  containerd:
[2021-02-19T14:53:51.850Z]   Version:          1.4.3
[2021-02-19T14:53:51.850Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-19T14:53:51.850Z]  runc:
[2021-02-19T14:53:51.850Z]   Version:          1.0.0-rc92
[2021-02-19T14:53:51.850Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-19T14:53:51.850Z]  docker-init:
[2021-02-19T14:53:51.850Z]   Version:          0.19.0
[2021-02-19T14:53:51.850Z]   GitCommit:        de40ad0
[2021-02-19T14:53:52.713Z] 3.4: Pulling from library/alpine
[2021-02-19T14:53:52.713Z] c1e54eec4b57: Pulling fs layer
[2021-02-19T14:53:52.972Z] c1e54eec4b57: Verifying Checksum
[2021-02-19T14:53:52.972Z] c1e54eec4b57: Download complete
[2021-02-19T14:53:53.231Z] c1e54eec4b57: Pull complete
[2021-02-19T14:53:53.231Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-02-19T14:53:53.231Z] Status: Downloaded newer image for alpine:3.4
[2021-02-19T14:53:55.438Z] + python .ci/scripts/pre_archive_test.py
[2021-02-19T14:53:56.412Z] Copy ./metricbeat/build into build/metricbeat/build
[2021-02-19T14:53:56.425Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24121/src/github.com/elastic/beats/build
[2021-02-19T14:53:56.731Z] + rm -rf ve
[2021-02-19T14:53:56.731Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-02-19T14:53:56.745Z] Recording test results
[2021-02-19T14:53:59.380Z] [Checks API] No suitable checks publisher found.
[2021-02-19T14:53:59.733Z] Client: Docker Engine - Community
[2021-02-19T14:53:59.733Z]  Version:           20.10.3
[2021-02-19T14:53:59.733Z]  API version:       1.41
[2021-02-19T14:53:59.733Z]  Go version:        go1.13.15
[2021-02-19T14:53:59.733Z]  Git commit:        48d30b5
[2021-02-19T14:53:59.733Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-19T14:53:59.733Z]  OS/Arch:           linux/amd64
[2021-02-19T14:53:59.733Z]  Context:           default
[2021-02-19T14:53:59.733Z]  Experimental:      true
[2021-02-19T14:53:59.733Z] 
[2021-02-19T14:53:59.733Z] Server: Docker Engine - Community
[2021-02-19T14:53:59.733Z]  Engine:
[2021-02-19T14:53:59.733Z]   Version:          20.10.3
[2021-02-19T14:53:59.733Z]   API version:      1.41 (minimum version 1.12)
[2021-02-19T14:53:59.733Z]   Go version:       go1.13.15
[2021-02-19T14:53:59.733Z]   Git commit:       46229ca
[2021-02-19T14:53:59.733Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-19T14:53:59.733Z]   OS/Arch:          linux/amd64
[2021-02-19T14:53:59.733Z]   Experimental:     false
[2021-02-19T14:53:59.733Z]  containerd:
[2021-02-19T14:53:59.733Z]   Version:          1.4.3
[2021-02-19T14:53:59.733Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-19T14:53:59.733Z]  runc:
[2021-02-19T14:53:59.733Z]   Version:          1.0.0-rc92
[2021-02-19T14:53:59.733Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-19T14:53:59.733Z]  docker-init:
[2021-02-19T14:53:59.733Z]   Version:          0.19.0
[2021-02-19T14:53:59.733Z]   GitCommit:        de40ad0
[2021-02-19T14:54:52.637Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-02-19T14:54:52.700Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24121/src/github.com/elastic/beats
[2021-02-19T14:54:53.259Z] Running on worker-395930 in /var/lib/jenkins/workspace/Beats_beats_PR-24121
[2021-02-19T14:54:53.307Z] [INFO] getVaultSecret: Getting secrets
[2021-02-19T14:54:53.396Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-02-19T14:54:55.405Z] + chmod 755 generate-build-data.sh
[2021-02-19T14:54:55.405Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24121/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24121/runs/5 FAILURE 2857713
[2021-02-19T14:54:55.405Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24121/runs/5/steps/?limit=10000 -o steps-info.json
[2021-02-19T14:55:01.785Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24121/runs/5/tests/?status=FAILED -o tests-errors.json
[2021-02-19T14:55:04.994Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24121/runs/5/log/ -o pipeline-log.txt

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 46694
Skipped 4922
Total 51616

@andresrc andresrc added the Team:Automation Label for the Observability productivity team label Feb 19, 2021
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 19, 2021
@v1v v1v removed the skip-ci Skip the build in the CI but linting label Feb 19, 2021
@v1v v1v changed the title [PACKAGING] Add multiplaform manifest [PACKAGING] Push docker images with the architecture in the version Feb 19, 2021
@v1v v1v requested a review from a team February 19, 2021 14:06
@v1v v1v marked this pull request as ready for review February 19, 2021 14:06
.ci/packaging.groovy Outdated Show resolved Hide resolved
// For backward compatibility let's ensure we tag only for amd64, then E2E can benefit from until
// they support the versioning with the architecture
if ("${arch}" == "amd64") {
doTagAndPush(beatName: beatName, variant: variant, sourceTag: libbetaVer, targetTag: "${tag}")
Copy link
Contributor

@mdelapenya mdelapenya Feb 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We addressed the change in elastic/e2e-testing#787

@v1v v1v merged commit daa4471 into elastic:master Feb 22, 2021
@v1v v1v deleted the feature/use-multiplatform-docker branch February 22, 2021 14:54
v1v added a commit to v1v/beats that referenced this pull request Feb 22, 2021
* upstream/master:
  [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155)
  [PACKAGING] Push docker images with the architecture in the version (elastic#24121)
  [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938)
  indicator type url is in upper case (elastic#24152)
  [Filebeat] Document netflow internal_networks and set default (elastic#24110)
  [Filebeat] Adding fixes to the TI module (elastic#24133)
  [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347)
  [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128)
  Set Elastic licence type for APM server Beats update job (elastic#24122)
  Add logrotation section on Running Filebeat on k8s (elastic#24120)
  [CI] Run if manual UI (elastic#24116)
  [CI] enable x-pack/heartbeat in the CI (elastic#23873)
v1v added a commit to v1v/beats that referenced this pull request Feb 23, 2021
…dows-7

* upstream/master:
  Remove OSS reference for kibana and elasticsearch (elastic#24164)
  Skip flaky TestActions on MacOSx (elastic#23966)
  [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null (elastic#24167)
  [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155)
  [PACKAGING] Push docker images with the architecture in the version (elastic#24121)
  [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938)
  indicator type url is in upper case (elastic#24152)
  [Filebeat] Document netflow internal_networks and set default (elastic#24110)
  [Filebeat] Adding fixes to the TI module (elastic#24133)
  [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347)
  [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128)
  Set Elastic licence type for APM server Beats update job (elastic#24122)
  Add logrotation section on Running Filebeat on k8s (elastic#24120)
  [CI] Run if manual UI (elastic#24116)
  [CI] enable x-pack/heartbeat in the CI (elastic#23873)
  chore: comment out the E2E (elastic#24109)
  chore: add-backport-next (elastic#24098)
  Adjust the position of the architecture name in Dockerlogbeat tarball (elastic#24095)
  Update dependencies for M1 support in System (elastic#24019)
v1v added a commit to v1v/beats that referenced this pull request Feb 23, 2021
…-arm

* upstream/master: (24 commits)
  Add example input autodsicover config (elastic#24157)
  Empty configuration options generate `<no value>` string for azure-eventhub input (elastic#24156)
  Remove OSS reference for kibana and elasticsearch (elastic#24164)
  Skip flaky TestActions on MacOSx (elastic#23966)
  [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null (elastic#24167)
  [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155)
  [PACKAGING] Push docker images with the architecture in the version (elastic#24121)
  [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938)
  indicator type url is in upper case (elastic#24152)
  [Filebeat] Document netflow internal_networks and set default (elastic#24110)
  [Filebeat] Adding fixes to the TI module (elastic#24133)
  [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347)
  [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128)
  Set Elastic licence type for APM server Beats update job (elastic#24122)
  Add logrotation section on Running Filebeat on k8s (elastic#24120)
  [CI] Run if manual UI (elastic#24116)
  [CI] enable x-pack/heartbeat in the CI (elastic#23873)
  chore: comment out the E2E (elastic#24109)
  chore: add-backport-next (elastic#24098)
  Adjust the position of the architecture name in Dockerlogbeat tarball (elastic#24095)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Automation Label for the Observability productivity team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants