Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ubuntu):Add an amazon-ebs AMI builder #1447

Merged
merged 17 commits into from
Oct 17, 2024

Conversation

jayfranco999
Copy link
Collaborator

@jayfranco999 jayfranco999 commented Oct 15, 2024

As per jenkins-infra/helpdesk#4353 (comment)

Related to jenkins-infra/helpdesk#4316 (comment)

This PR introduces the following to packer-images:

  • Add amazon-ebs builder logic in Jenkinsfile_k8s
  • Include EC2 source for build-jenkins-agent-ubuntu.pkr.hcl
  • Add datasources.pkr.hcl to look-up ubuntu-22_04 AMI id for respective architecture
  • Define aws_instance_types in locals.pkr.hcl
  • Include Amazon plugin for packer-images
  • Define all the common settings for AWS base image AMI's in sources.pkr.hcl
  • Add AWS related packer-image variables (AWS region and credentials)

Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
variables.pkr.hcl Outdated Show resolved Hide resolved
sources.pkr.hcl Outdated Show resolved Hide resolved
datasources.pkr.hcl Outdated Show resolved Hide resolved
datasources.pkr.hcl Outdated Show resolved Hide resolved
Copy link
Contributor

@dduportal dduportal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jayfranco999 jayfranco999 marked this pull request as draft October 15, 2024 07:51
jayfranco999 and others added 2 commits October 15, 2024 17:38
Co-authored-by: Damien Duportal <damien.duportal@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
@jayfranco999
Copy link
Collaborator Author

jayfranco999 commented Oct 15, 2024

Update:

ubuntu-22_04 AMD64 and ARM64 AMI images were successfully built with packer-images – https://infra.ci.jenkins.io/job/infra-tools/job/packer-images/job/PR-1447/5/

Since windows AMI image source logic is not included in this PR, the windows agent builds are failing in PACKER_LOG=1 packer validate ./ stage of the build pipeline with exit-code-1

[2024-10-15T12:27:00.527Z]   on sources.pkr.hcl line 33:
[2024-10-15T12:27:00.527Z]   (source code not available)
[2024-10-15T12:27:00.527Z] 
[2024-10-15T12:27:00.527Z] with data.amazon-ami as map of object with 1 element,
[2024-10-15T12:27:00.527Z]      local.agent_os_version_safe as "2022",
[2024-10-15T12:27:00.527Z]      var.agent_os_type as "windows".
[2024-10-15T12:27:00.527Z] 
[2024-10-15T12:27:00.527Z] The given key does not identify an element in this collection value.
[2024-10-15T12:27:00.527Z] 
[2024-10-15T12:27:00.527Z] Error: Invalid index
[2024-10-15T12:27:00.527Z] 
[2024-10-15T12:27:00.527Z]   on sources.pkr.hcl line 33:
[2024-10-15T12:27:00.527Z]   (source code not available)
[2024-10-15T12:27:00.527Z] 
[2024-10-15T12:27:00.527Z] with data.amazon-ami as map of object with 1 element,
[2024-10-15T12:27:00.527Z]      local.agent_os_version_safe as "2022",
[2024-10-15T12:27:00.527Z]      var.agent_os_type as "windows".
[2024-10-15T12:27:00.527Z] 
[2024-10-15T12:27:00.527Z] The given key does not identify an element in this collection value.

I believe the problem is with source_ami = data.amazon-ami["${var.agent_os_type}-${local.agent_os_version_safe}"].id command in sources.pkr.hcl that causes errors in the packer validate stage.

Further steps will include exporting the aws credentials and region for terraform-packer-user as environment variables instead of using static variables.

Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
Jenkinsfile_k8s Outdated Show resolved Hide resolved
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
locals.pkr.hcl Outdated Show resolved Hide resolved
sources.pkr.hcl Outdated Show resolved Hide resolved
sources.pkr.hcl Outdated Show resolved Hide resolved
sources.pkr.hcl Outdated Show resolved Hide resolved
variables.pkr.hcl Outdated Show resolved Hide resolved
@dduportal
Copy link
Contributor

@jayfranco999 Proposal for a clear title:

feat(ubuntu):Add an `amazon-ebs` AMI builder

instead of:

feat!(aws):Add Amazon-ebs builder for Linux AMIs
  • It is a new feature, not a breaking change at all
  • AWS is not a component of the repository. Ubuntu is.

WDYT?

@jayfranco999 jayfranco999 changed the title feat!(aws):Add Amazon-ebs builder for Linux AMIs feat(ubuntu):Add an amazon-ebs AMI builder Oct 16, 2024
Signed-off-by: jayfranco999 <jaideepfrancis999@gmail.com>
@jayfranco999 jayfranco999 marked this pull request as ready for review October 16, 2024 15:40
@jayfranco999 jayfranco999 dismissed dduportal’s stale review October 17, 2024 12:41

Addressed with latest commit

Jenkinsfile_k8s Outdated Show resolved Hide resolved
@dduportal dduportal merged commit ccf81f7 into jenkins-infra:main Oct 17, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants