Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: secret copy #741 #948

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .codespellrc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Lint Codespell configurations
[codespell]
skip = .codespellrc,.git,node_modules,build,dist,*.zst,CHANGELOG.md,.playwright,.terraform
ignore-words-list = NotIn,AKS,LICENS,aks
ignore-words-list = NotIn,AKS,LICENS,aks,afterAll
enable-colors =
check-hidden =
10 changes: 10 additions & 0 deletions src/pepr/operator/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ import { purgeAuthserviceClients } from "./controllers/keycloak/authservice/auth
import { exemptValidator } from "./crd/validators/exempt-validator";
import { packageReconciler } from "./reconcilers/package-reconciler";

// Secret imports
import { copySecret, labelCopySecret, validateSecret } from "./secrets";

// Export the operator capability for registration in the root pepr.ts
export { operator } from "./common";

Expand Down Expand Up @@ -91,3 +94,10 @@ When(UDSPackage)
log.info("Identity and Authorization layer removed, operator will NOT handle SSO.");
UDSConfig.isIdentityDeployed = false;
});

// Watch for secrets w/ the UDS secret label and copy as necessary
When(a.Secret)
.IsCreatedOrUpdated()
.WithLabel(labelCopySecret, "true")
.Mutate(request => copySecret(request))
.Validate(request => validateSecret(request));
Comment on lines +102 to +103
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be curious to get your thoughts on this design approach of mutating the "destination" secret. Initially I thought this was a mistake because I didn't read far enough in the PR 😅.

My inclination would be that it makes more sense to Watch (or Reconcile) secrets that have a label like uds.dev/copyMe (i.e. the "source" secret). That way you can guarantee you know when the secrets update at the source and ensure they are kept up to date in the destination. This current pattern would only watch changes/creations on the destination side, meaning you might miss changes if the source updates I think? There may be a few more design implementations to think through here, but overall I think it would make a bit more sense? (a few of those design decisions: do you need an overwrite: true/false annotation, do you support mutliple destinations and how, etc)

I think this might also resolve most of @bburky's security concerns since the only secret being read would be one that has been explicitly labelled to be copied. The only caveat would be on that overwrite piece - we would want to be careful implementing this to not overprivilege that label to write to any secret without some safeguards (maybe we only allow overwriting if pepr is already controlling the secret, use ownership metadata, etc).

The one obvious downside I could see to this is that you would need to have access to label/annotate the source secret which might be an issue with some upstream charts/operators? If that is a major sticking point we could move to a custom resource as @bburky recommended. If we want to go that route I think I'd lean slightly towards a new custom resource rather than using the Package CR.

175 changes: 175 additions & 0 deletions src/pepr/operator/secrets.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
/**
* Copyright 2024 Defense Unicorns
* SPDX-License-Identifier: AGPL-3.0-or-later OR LicenseRef-Defense-Unicorns-Commercial
*/

import { afterAll, beforeAll, describe, expect, it } from "@jest/globals";
import { K8s, kind } from "pepr";

const failIfReached = () => expect(true).toBe(false);

describe("test secret copy", () => {
const sourceSecret = {
metadata: {
name: "source-secret",
namespace: "source-namespace",
},
data: { key: "TESTCASE" },
};

beforeAll(async () => {
// Setup test namespaces
await K8s(kind.Namespace).Apply({
metadata: { name: "source-namespace" },
});
await K8s(kind.Namespace).Apply({
metadata: { name: "destination-namespace" },
});
await K8s(kind.Namespace).Apply({
metadata: { name: "destination-namespace2" },
});
await K8s(kind.Namespace).Apply({
metadata: { name: "destination-namespace3" },
});

// Create source secret
await K8s(kind.Secret).Apply(sourceSecret);
});

afterAll(async () => {
// Cleanup test namespaces
await K8s(kind.Namespace).Delete("source-namespace");
await K8s(kind.Namespace).Delete("destination-namespace");
await K8s(kind.Namespace).Delete("destination-namespace2");
// await K8s(kind.Namespace).Delete("destination-namespace3");
});

it("should copy a secret with the secrets.uds.dev/copy label", async () => {
// Apply destination secret
const destinationSecret = {
metadata: {
name: "destination-secret",
namespace: "destination-namespace",
labels: { "secrets.uds.dev/copy": "true" },
annotations: {
"secrets.uds.dev/fromNamespace": "source-namespace",
"secrets.uds.dev/fromName": "source-secret",
},
},
};

// Check if destination secret has the same data as the source secret
const destSecret = await K8s(kind.Secret).Apply(destinationSecret);
expect(destSecret.data).toEqual({ key: "VEVTVENBU0U=" }); // base64 encoded "TESTCASE"

// Confirm that label has changed from copy to copied
expect(destSecret.metadata?.labels).toEqual({
"secrets.uds.dev/copied": "true",
});
});

it("should not copy a secret without the secrets.uds.dev/copy=true label", async () => {
// Apply destination secret
const destinationSecret1 = {
metadata: {
name: "destination-secret-tc2a",
namespace: "destination-namespace2",
labels: { "secrets.uds.dev/copy": "false" },
annotations: {
"secrets.uds.dev/fromNamespace": "source-namespace",
"secrets.uds.dev/fromName": "source-secret",
},
},
};

const destinationSecret2 = {
metadata: {
name: "destination-secret-tc2b",
namespace: "destination-namespace2",
labels: { asdf: "true" },
annotations: {
"secrets.uds.dev/fromNamespace": "source-namespace",
"secrets.uds.dev/fromName": "source-secret",
},
},
};

const destSecret1 = await K8s(kind.Secret).Apply(destinationSecret1);
const destSecret2 = await K8s(kind.Secret).Apply(destinationSecret2);

// Confirm destination secrets are created "as is"
expect(destSecret1.data).toEqual(undefined);
expect(destSecret1.metadata?.labels).toEqual({
"secrets.uds.dev/copy": "false",
});

expect(destSecret2.data).toEqual(undefined);
expect(destSecret2.metadata?.labels).toEqual({ asdf: "true" });
});

it("should error when copy label is present but missing annotations", async () => {
const destinationSecret = {
metadata: {
name: "destination-secret-tc3",
namespace: "destination-namespace3",
labels: { "secrets.uds.dev/copy": "true" },
},
};

const expected = (e: Error) => {
expect(e).toMatchObject({
ok: false,
data: {
message: expect.stringContaining("denied the request"),
},
});
};

return K8s(kind.Secret).Apply(destinationSecret).then(failIfReached).catch(expected);
});

it("should error when missing source secret and onMissingSource=Deny", async () => {
const destinationSecret = {
metadata: {
name: "destination-secret",
namespace: "destination-namespace",
labels: { "secrets.uds.dev/copy": "true" },
annotations: {
"secrets.uds.dev/fromNamespace": "missing-namespace",
"secrets.uds.dev/fromName": "missing-secret",
"secrets.uds.dev/onMissingSource": "Deny",
},
},
};

const expected = (e: Error) => {
expect(e).toMatchObject({
ok: false,
data: {
message: expect.stringContaining("denied the request"),
},
});
};

return K8s(kind.Secret).Apply(destinationSecret).then(failIfReached).catch(expected);
});

it("should create empty secret when missing source secret and onMissingSource=LeaveEmpty", async () => {
const destinationSecret = {
metadata: {
name: "destination-secret-tc4a",
namespace: "destination-namespace",
labels: { "secrets.uds.dev/copy": "true" },
annotations: {
"secrets.uds.dev/fromNamespace": "source-namespace",
"secrets.uds.dev/fromName": "missing-secret",
"secrets.uds.dev/onMissingSource": "LeaveEmpty",
},
},
};

const destSecret = await K8s(kind.Secret).Apply(destinationSecret);

expect(destSecret.data).toEqual(undefined);
});
});
Loading