Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDK Migration - V2 Beta #306

Closed
wants to merge 19 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions .github/workflows/wip-test-oidc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
# taken and modified from https://github.com/hashicorp/go-azure-sdk/blob/main/.github/workflows/pr-acceptance-tests.yml
name: OIDC Example - Testing OIDC integration in the SDK branch
on:
push:

permissions:
contents: read
id-token: write

jobs:
secrets-check:
runs-on: ubuntu-latest
outputs:
available: "${{ steps.check-secrets.outputs.available }}"
steps:
# we check for the ACTIONS_ID_TOKEN_REQUEST_URL variable as a proxy for other secrets
# it will be unset when running for a PR from a fork
- id: check-secrets
run: |
if [[ "${ACTIONS_ID_TOKEN_REQUEST_URL}" == "" ]]; then
echo "available=false" | tee ${GITHUB_OUTPUT}
else
echo "available=true" | tee ${GITHUB_OUTPUT}
fi

test-oidc:
runs-on: ubuntu-latest
needs: [secrets-check]
if: needs.secrets-check.outputs.available == 'true'
steps:
- name: Set OIDC Token
run: |
echo "ARM_OIDC_TOKEN=$(curl -H "Accept: application/json; api-version=2.0" -H "Authorization: Bearer ${ACTIONS_ID_TOKEN_REQUEST_TOKEN}" -H "Content-Type: application/json" -G --data-urlencode "audience=api://AzureADTokenExchange" "${ACTIONS_ID_TOKEN_REQUEST_URL}" | jq -r '.value')" >>${GITHUB_ENV}

- name: Install Go
uses: actions/setup-go@4d34df0c2316fe8122ab82dc22947d607c0c91f9 # v4.0.0
with:
go-version: '1.19.5'

- name: Checkout
uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2

- name: Setup `packer`
uses: hashicorp/setup-packer@main
id: setup

- name: Build the plugin
run: make

- name: Try to run an AzureARM build with our OIDC token
run: packer build -force ./example/oidc-example.pkr.hcl
env:
ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID}}
ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID}}
30 changes: 6 additions & 24 deletions builder/azure/arm/artifact.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ const (
)

type AdditionalDiskArtifact struct {
AdditionalDiskUri string
AdditionalDiskUriReadOnlySas string
AdditionalDiskUri string
}

type Artifact struct {
Expand All @@ -32,8 +31,6 @@ type Artifact struct {
StorageAccountLocation string
OSDiskUri string
TemplateUri string
OSDiskUriReadOnlySas string
TemplateUriReadOnlySas string

// Managed Image
ManagedImageResourceGroupName string
Expand All @@ -56,7 +53,7 @@ type Artifact struct {
StateData map[string]interface{}
}

func NewManagedImageArtifact(osType, resourceGroup, name, location, id, osDiskSnapshotName, dataDiskSnapshotPrefix string, generatedData map[string]interface{}, keepOSDisk bool, template *CaptureTemplate, getSasUrl func(name string) string) (*Artifact, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very ignorant of this plugin, but what's a SAS URL, and why do we remove that here? I presume this is not necessary anymore?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related but not too much, I see the only other place where there's a reference to a getSasUrl is in the NewArtifact function for the dtl builder (in builder/azure/dtl/artifact.go line 56), which is never used in the code according to gopls, so I presume we could remove this as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, a SAS URL is a link to an Azure Storage account resource with an embedded token, we create SAS tokens in VHD builds and when using managed image builds without deleting the disk I believe, however we've been asked by a lot of our users to remove this here #250, because of the security concerns of using these they are not recommended by Microsoft, we are doing them in every build. To top it off the new SDK does not provide the same helper we currently use to generate these tokens.

So for v2 after chatting with Wilken, and given the above issue, I am proposing removing them, they can still be created by users outside of Packer, and this was never used in the resulting Azure Artifact, but was simply printed in the terminal output of the build.

Yes we should remove it from the DTL builder but I would like to hold of any changes to the DTL builder until we have signed off on the way we've handling this for the ARM builder

func NewManagedImageArtifact(osType, resourceGroup, name, location, id, osDiskSnapshotName, dataDiskSnapshotPrefix string, generatedData map[string]interface{}, keepOSDisk bool, template *CaptureTemplate) (*Artifact, error) {
res := Artifact{
ManagedImageResourceGroupName: resourceGroup,
ManagedImageName: name,
Expand Down Expand Up @@ -86,7 +83,6 @@ func NewManagedImageArtifact(osType, resourceGroup, name, location, id, osDiskSn
}

res.OSDiskUri = vhdUri.String()
res.OSDiskUriReadOnlySas = getSasUrl(getStorageUrlPath(vhdUri))
}

return &res, nil
Expand Down Expand Up @@ -115,7 +111,7 @@ func NewSharedImageArtifact(osType, destinationSharedImageGalleryId string, loca
}, nil
}

func NewArtifact(template *CaptureTemplate, getSasUrl func(name string) string, osType string, generatedData map[string]interface{}) (*Artifact, error) {
func NewArtifact(template *CaptureTemplate, osType string, generatedData map[string]interface{}) (*Artifact, error) {
if template == nil {
return nil, fmt.Errorf("nil capture template")
}
Expand Down Expand Up @@ -143,17 +139,14 @@ func NewArtifact(template *CaptureTemplate, getSasUrl func(name string) string,
return nil, err
}
data_disks[i].AdditionalDiskUri = additionalVhdUri.String()
data_disks[i].AdditionalDiskUriReadOnlySas = getSasUrl(getStorageUrlPath(additionalVhdUri))
}
additional_disks = &data_disks
}

return &Artifact{
OSType: osType,
OSDiskUri: vhdUri.String(),
OSDiskUriReadOnlySas: getSasUrl(getStorageUrlPath(vhdUri)),
TemplateUri: templateUri.String(),
TemplateUriReadOnlySas: getSasUrl(getStorageUrlPath(templateUri)),
OSType: osType,
OSDiskUri: vhdUri.String(),
TemplateUri: templateUri.String(),

AdditionalDisks: additional_disks,

Expand All @@ -163,11 +156,6 @@ func NewArtifact(template *CaptureTemplate, getSasUrl func(name string) string,
}, nil
}

func getStorageUrlPath(u *url.URL) string {
parts := strings.Split(u.Path, "/")
return strings.Join(parts[3:], "/")
}

func storageUriToTemplateUri(su *url.URL) (*url.URL, error) {
// packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd -> 4085bb15-3644-4641-b9cd-f575918640b4
filename := path.Base(su.Path)
Expand Down Expand Up @@ -249,19 +237,13 @@ func (a *Artifact) String() string {
if a.OSDiskUri != "" {
buf.WriteString(fmt.Sprintf("OSDiskUri: %s\n", a.OSDiskUri))
}
if a.OSDiskUriReadOnlySas != "" {
buf.WriteString(fmt.Sprintf("OSDiskUriReadOnlySas: %s\n", a.OSDiskUriReadOnlySas))
}
} else if !a.isPublishedToSIG() {
buf.WriteString(fmt.Sprintf("StorageAccountLocation: %s\n", a.StorageAccountLocation))
buf.WriteString(fmt.Sprintf("OSDiskUri: %s\n", a.OSDiskUri))
buf.WriteString(fmt.Sprintf("OSDiskUriReadOnlySas: %s\n", a.OSDiskUriReadOnlySas))
buf.WriteString(fmt.Sprintf("TemplateUri: %s\n", a.TemplateUri))
buf.WriteString(fmt.Sprintf("TemplateUriReadOnlySas: %s\n", a.TemplateUriReadOnlySas))
if a.AdditionalDisks != nil {
for i, additionaldisk := range *a.AdditionalDisks {
buf.WriteString(fmt.Sprintf("AdditionalDiskUri (datadisk-%d): %s\n", i+1, additionaldisk.AdditionalDiskUri))
buf.WriteString(fmt.Sprintf("AdditionalDiskUriReadOnlySas (datadisk-%d): %s\n", i+1, additionaldisk.AdditionalDiskUriReadOnlySas))
}
}
}
Expand Down
60 changes: 12 additions & 48 deletions builder/azure/arm/artifact_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
package arm

import (
"fmt"
"strings"
"testing"

Expand All @@ -13,10 +12,6 @@ import (
"github.com/mitchellh/mapstructure"
)

func getFakeSasUrl(name string) string {
return fmt.Sprintf("SAS-%s", name)
}

func generatedData() map[string]interface{} {
return make(map[string]interface{})
}
Expand All @@ -39,7 +34,7 @@ func TestArtifactIdVHD(t *testing.T) {
},
}

artifact, err := NewArtifact(&template, getFakeSasUrl, "Linux", generatedData())
artifact, err := NewArtifact(&template, "Linux", generatedData())
if err != nil {
t.Fatalf("err=%s", err)
}
Expand Down Expand Up @@ -70,7 +65,7 @@ func TestArtifactIDManagedImage(t *testing.T) {
},
}

artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "fakeDataDiskSnapshotPrefix", generatedData(), false, &template, getFakeSasUrl)
artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "fakeDataDiskSnapshotPrefix", generatedData(), false, &template)
if err != nil {
t.Fatalf("err=%s", err)
}
Expand Down Expand Up @@ -110,7 +105,7 @@ func TestArtifactIDManagedImageWithoutOSDiskSnapshotName(t *testing.T) {
},
}

artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "", "fakeDataDiskSnapshotPrefix", generatedData(), false, &template, getFakeSasUrl)
artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "", "fakeDataDiskSnapshotPrefix", generatedData(), false, &template)
if err != nil {
t.Fatalf("err=%s", err)
}
Expand Down Expand Up @@ -149,7 +144,7 @@ func TestArtifactIDManagedImageWithoutDataDiskSnapshotPrefix(t *testing.T) {
},
}

artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "", generatedData(), false, &template, getFakeSasUrl)
artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "", generatedData(), false, &template)
if err != nil {
t.Fatalf("err=%s", err)
}
Expand Down Expand Up @@ -188,7 +183,7 @@ func TestArtifactIDManagedImageWithKeepingTheOSDisk(t *testing.T) {
},
}

artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "", generatedData(), true, &template, getFakeSasUrl)
artifact, err := NewManagedImageArtifact("Linux", "fakeResourceGroup", "fakeName", "fakeLocation", "fakeID", "fakeOsDiskSnapshotName", "", generatedData(), true, &template)
if err != nil {
t.Fatalf("err=%s", err)
}
Expand All @@ -202,7 +197,6 @@ ManagedImageId: fakeID
ManagedImageLocation: fakeLocation
ManagedImageOSDiskSnapshotName: fakeOsDiskSnapshotName
OSDiskUri: https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd
OSDiskUriReadOnlySas: SAS-Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd
`

result := artifact.String()
Expand Down Expand Up @@ -414,7 +408,7 @@ func TestArtifactString(t *testing.T) {
},
}

artifact, err := NewArtifact(&template, getFakeSasUrl, "Linux", generatedData())
artifact, err := NewArtifact(&template, "Linux", generatedData())
if err != nil {
t.Fatalf("err=%s", err)
}
Expand All @@ -423,15 +417,9 @@ func TestArtifactString(t *testing.T) {
if !strings.Contains(testSubject, "OSDiskUri: https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd") {
t.Errorf("Expected String() output to contain OSDiskUri")
}
if !strings.Contains(testSubject, "OSDiskUriReadOnlySas: SAS-Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd") {
t.Errorf("Expected String() output to contain OSDiskUriReadOnlySas")
}
if !strings.Contains(testSubject, "TemplateUri: https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json") {
t.Errorf("Expected String() output to contain TemplateUri")
}
if !strings.Contains(testSubject, "TemplateUriReadOnlySas: SAS-Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json") {
t.Errorf("Expected String() output to contain TemplateUriReadOnlySas")
}
if !strings.Contains(testSubject, "StorageAccountLocation: southcentralus") {
t.Errorf("Expected String() output to contain StorageAccountLocation")
}
Expand Down Expand Up @@ -465,7 +453,7 @@ func TestAdditionalDiskArtifactString(t *testing.T) {
},
}

artifact, err := NewArtifact(&template, getFakeSasUrl, "Linux", generatedData())
artifact, err := NewArtifact(&template, "Linux", generatedData())
if err != nil {
t.Fatalf("err=%s", err)
}
Expand All @@ -474,15 +462,9 @@ func TestAdditionalDiskArtifactString(t *testing.T) {
if !strings.Contains(testSubject, "OSDiskUri: https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd") {
t.Errorf("Expected String() output to contain OSDiskUri")
}
if !strings.Contains(testSubject, "OSDiskUriReadOnlySas: SAS-Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd") {
t.Errorf("Expected String() output to contain OSDiskUriReadOnlySas")
}
if !strings.Contains(testSubject, "TemplateUri: https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json") {
t.Errorf("Expected String() output to contain TemplateUri")
}
if !strings.Contains(testSubject, "TemplateUriReadOnlySas: SAS-Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json") {
t.Errorf("Expected String() output to contain TemplateUriReadOnlySas")
}
if !strings.Contains(testSubject, "StorageAccountLocation: southcentralus") {
t.Errorf("Expected String() output to contain StorageAccountLocation")
}
Expand All @@ -492,9 +474,6 @@ func TestAdditionalDiskArtifactString(t *testing.T) {
if !strings.Contains(testSubject, "AdditionalDiskUri (datadisk-1): https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-datadisk-1.4085bb15-3644-4641-b9cd-f575918640b4.vhd") {
t.Errorf("Expected String() output to contain AdditionalDiskUri")
}
if !strings.Contains(testSubject, "AdditionalDiskUriReadOnlySas (datadisk-1): SAS-Images/images/packer-datadisk-1.4085bb15-3644-4641-b9cd-f575918640b4.vhd") {
t.Errorf("Expected String() output to contain AdditionalDiskUriReadOnlySas")
}
}

func TestArtifactProperties(t *testing.T) {
Expand All @@ -515,23 +494,17 @@ func TestArtifactProperties(t *testing.T) {
},
}

testSubject, err := NewArtifact(&template, getFakeSasUrl, "Linux", generatedData())
testSubject, err := NewArtifact(&template, "Linux", generatedData())
if err != nil {
t.Fatalf("err=%s", err)
}

if testSubject.OSDiskUri != "https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd" {
t.Errorf("Expected template to be 'https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd', but got %s", testSubject.OSDiskUri)
}
if testSubject.OSDiskUriReadOnlySas != "SAS-Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd" {
t.Errorf("Expected template to be 'SAS-Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd', but got %s", testSubject.OSDiskUriReadOnlySas)
}
if testSubject.TemplateUri != "https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json" {
t.Errorf("Expected template to be 'https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json', but got %s", testSubject.TemplateUri)
}
if testSubject.TemplateUriReadOnlySas != "SAS-Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json" {
t.Errorf("Expected template to be 'SAS-Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json', but got %s", testSubject.TemplateUriReadOnlySas)
}
if testSubject.StorageAccountLocation != "southcentralus" {
t.Errorf("Expected StorageAccountLocation to be 'southcentral', but got %s", testSubject.StorageAccountLocation)
}
Expand Down Expand Up @@ -565,23 +538,17 @@ func TestAdditionalDiskArtifactProperties(t *testing.T) {
},
}

testSubject, err := NewArtifact(&template, getFakeSasUrl, "Linux", generatedData())
testSubject, err := NewArtifact(&template, "Linux", generatedData())
if err != nil {
t.Fatalf("err=%s", err)
}

if testSubject.OSDiskUri != "https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd" {
t.Errorf("Expected template to be 'https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd', but got %s", testSubject.OSDiskUri)
}
if testSubject.OSDiskUriReadOnlySas != "SAS-Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd" {
t.Errorf("Expected template to be 'SAS-Images/images/packer-osDisk.4085bb15-3644-4641-b9cd-f575918640b4.vhd', but got %s", testSubject.OSDiskUriReadOnlySas)
}
if testSubject.TemplateUri != "https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json" {
t.Errorf("Expected template to be 'https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json', but got %s", testSubject.TemplateUri)
}
if testSubject.TemplateUriReadOnlySas != "SAS-Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json" {
t.Errorf("Expected template to be 'SAS-Images/images/packer-vmTemplate.4085bb15-3644-4641-b9cd-f575918640b4.json', but got %s", testSubject.TemplateUriReadOnlySas)
}
if testSubject.StorageAccountLocation != "southcentralus" {
t.Errorf("Expected StorageAccountLocation to be 'southcentral', but got %s", testSubject.StorageAccountLocation)
}
Expand All @@ -597,9 +564,6 @@ func TestAdditionalDiskArtifactProperties(t *testing.T) {
if (*testSubject.AdditionalDisks)[0].AdditionalDiskUri != "https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-datadisk-1.4085bb15-3644-4641-b9cd-f575918640b4.vhd" {
t.Errorf("Expected additional disk uri to be 'https://storage.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-datadisk-1.4085bb15-3644-4641-b9cd-f575918640b4.vhd', but got %s", (*testSubject.AdditionalDisks)[0].AdditionalDiskUri)
}
if (*testSubject.AdditionalDisks)[0].AdditionalDiskUriReadOnlySas != "SAS-Images/images/packer-datadisk-1.4085bb15-3644-4641-b9cd-f575918640b4.vhd" {
t.Errorf("Expected additional disk sas to be 'SAS-Images/images/packer-datadisk-1.4085bb15-3644-4641-b9cd-f575918640b4.vhd', but got %s", (*testSubject.AdditionalDisks)[0].AdditionalDiskUriReadOnlySas)
}
}

func TestArtifactOverHyphenatedCaptureUri(t *testing.T) {
Expand All @@ -620,7 +584,7 @@ func TestArtifactOverHyphenatedCaptureUri(t *testing.T) {
},
}

testSubject, err := NewArtifact(&template, getFakeSasUrl, "Linux", generatedData())
testSubject, err := NewArtifact(&template, "Linux", generatedData())
if err != nil {
t.Fatalf("err=%s", err)
}
Expand All @@ -633,7 +597,7 @@ func TestArtifactOverHyphenatedCaptureUri(t *testing.T) {
func TestArtifactRejectMalformedTemplates(t *testing.T) {
template := CaptureTemplate{}

_, err := NewArtifact(&template, getFakeSasUrl, "Linux", generatedData())
_, err := NewArtifact(&template, "Linux", generatedData())
if err == nil {
t.Fatalf("Expected artifact creation to fail, but it succeeded.")
}
Expand All @@ -656,7 +620,7 @@ func TestArtifactRejectMalformedStorageUri(t *testing.T) {
},
}

_, err := NewArtifact(&template, getFakeSasUrl, "Linux", generatedData())
_, err := NewArtifact(&template, "Linux", generatedData())
if err == nil {
t.Fatalf("Expected artifact creation to fail, but it succeeded.")
}
Expand Down
Loading