Skip to content

Commit

Permalink
feat(template)!: introducing variables and sensitive configuration (#672
Browse files Browse the repository at this point in the history
)

* feat(template): sample sensitive config

* feat(template): updated funcs, tests

* fix: go mod tidy

* fix: small refactor on template render

* docs: updated config/template docs

* fix: bump timeout in tui tests

* fix: small refactor for extending tmpl

* fix: bumped timeout

* fix: updated docs, slightly modified templaterenderer struct
  • Loading branch information
meganwolf0 authored Sep 25, 2024
1 parent 7b9a771 commit 5d1f232
Show file tree
Hide file tree
Showing 26 changed files with 1,628 additions and 178 deletions.
13 changes: 11 additions & 2 deletions docs/cli-commands/lula_tools_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,20 @@ lula tools template [flags]

```
To template an OSCAL Model:
To template an OSCAL Model, defaults to masking sensitive variables:
lula tools template -f ./oscal-component.yaml
To indicate a specific output file:
lula tools template -f ./oscal-component.yaml -o templated-oscal-component.yaml
Data for the templating should be stored under the 'variables' configuration item in a lula-config.yaml file
To perform overrides on the template data:
lula tools template -f ./oscal-component.yaml --set .var.key1=value1 --set .const.key2=value2
To perform the full template operation, including sensitive data:
lula tools template -f ./oscal-component.yaml --render all
Data for templating should be stored under 'constants' or 'variables' configuration items in a lula-config.yaml file
See documentation for more detail on configuration schema
```

Expand All @@ -35,6 +42,8 @@ Data for the templating should be stored under the 'variables' configuration ite
-h, --help help for template
-f, --input-file string the path to the target artifact
-o, --output-file string the path to the output file. If not specified, the output file will be directed to stdout
-r, --render string values to render the template with, options are: masked, constants, non-sensitive, all (default "masked")
-s, --set strings set a value in the template data
```

### Options inherited from parent commands
Expand Down
6 changes: 5 additions & 1 deletion docs/getting-started/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,8 @@ See the following tutorials for some introductory lessons on how to use Lula. If

Lula Validation manifests are the underlying mechanisms that dictates the evaluation of a system against a control as resulting in `satisfied` or `not satisfied`. A Lula Validation is linked to a control within a component definition via the OSCAL-specific property, [links](../oscal/oscal-validation-links.md).

Developing Lula Validations can sometimes be more art than science, but generally they should aim to be clear, concise, and robust to system changes. See our guide for [developing Lula Validations](./develop-a-validation.md) and the [references](../reference/README.md) for additional information.
Developing Lula Validations can sometimes be more art than science, but generally they should aim to be clear, concise, and robust to system changes. See our guide for [developing Lula Validations](./develop-a-validation.md) and the [references](../reference/README.md) for additional information.

### Configuration

Lula supports the addition of a configuration file for specifying CLI flags and templating values. See our [configuration](./configuration.md) guide for more information.
139 changes: 139 additions & 0 deletions docs/getting-started/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# Configuration

Lula allows the use and specification of a config file in the following ways:
- Checking current working directory for a `lula-config.yaml` file
- Specification with environment variable `LULA_CONFIG=<path>`

Environment Variables can be used to specify configuration values through use of `LULA_<VAR>` -> Example: `LULA_TARGET=il5`

## Identification

If identified, Lula will log which configuration file is used to stdout:
```bash
Using config file /home/dev/work/lula/lula-config.yaml
```

## Precedence

The precedence for configuring settings, such as `target`, follows this hierarchy:

### **Command Line Flag > Environment Variable > Configuration File**

1. **Command Line Flag:**
When a setting like `target` is specified using a command line flag, this value takes the highest precedence, overriding any environment variable or configuration file settings.

2. **Environment Variable:**
If the setting is not provided via a command line flag, an environment variable (e.g., `export LULA_TARGET=il5`) will take precedence over the configuration file.

3. **Configuration File:**
In the absence of both a command line flag and environment variable, the value specified in the configuration file will be used. This will override system defaults.

## Support

Modification of command variables can be set in the configuration file:

lula-config.yaml
```yaml
log_level: debug
target: il4
summary: true
```
### Templating Configuration Fields
Templating values are set in the configuration file via the use of `constants` and `variables` fields.

#### Constants

A sample `constants` section of a `lula-config.yaml` file is as follows:

```yaml
constants:
type: software
title: lula
resources:
name: test-pod-label
namespace: validation-test
imagelist:
- nginx
- nginx2
```

Constants will respect the structure of a map[string]interface{} and can be referenced as follows:

```yaml
# validaiton.yaml
metadata:
name: sample {{ .const.type }} validation for {{ .const.title }}
domain:
type: kubernetes
kubernetes-spec:
resources:
- name: myPod
resource-rule:
name: {{ .const.resources.name }}
version: v1
resource: pods
namespaces: [{{ .const.resources.namespace }}]
provider:
type: opa
opa-spec:
rego: |
package validate
import rego.v1
validate if {
input.myPod.metadata.name == {{ .const.resources.name }}
input.myPod.containers[_].name in { {{ .const.resources.imagelist | concatToRegoList }} }
}
```

And will be rendered as:
```yaml
metadata:
name: sample software validation for lula
domain:
type: kubernetes
kubernetes-spec:
resources:
- name: myPod
resource-rule:
name: myPod
version: v1
resource: pods
namespaces: [validation-test]
provider:
type: opa
opa-spec:
rego: |
package validate
import rego.v1
validate if {
input.myPod.metadata.name == "myPod"
input.myPod.containers[_].image in { "nginx", "nginx2" }
}
```

The constant's keys should be in the format `.const.<key>` and should not contain any '-' or '.' characters, as this will not respect the go text/template format.

#### Variables

A sample `variables` section of a `lula-config.yaml` file is as follows:

```yaml
variables:
- key: some_lula_secret
sensitive: true
- key: some_env_var
default: this-should-be-overridden
```

The `variables` section is a list of `key`, `default`, and `sensitive` fields, where `sensitive` and `default` are optional. The `key` and `default` fields are strings, and the `sensitive` field is a boolean.

A default value can be specified in the case where an environment variable may or may not be set, however an environment variable will always take precedence over a default value.

The environment variable should follow the pattern of `LULA_VAR_<key>` (not case sensitive), where `<key>` is the key specified in the `variables` section.

When using `sensitive` variables, the default behavior is to mask the value in the output of the template.
40 changes: 0 additions & 40 deletions docs/reference/configuration.md

This file was deleted.

3 changes: 0 additions & 3 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ require (
github.com/charmbracelet/x/exp/teatest v0.0.0-20240919170804-a4978c8e603a
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
github.com/defenseunicorns/go-oscal v0.6.0
github.com/defenseunicorns/pkg/helpers v1.1.3
github.com/hashicorp/go-version v1.7.0
github.com/kyverno/kyverno-json v0.0.3
github.com/mattn/go-runewidth v0.0.16
Expand Down Expand Up @@ -114,7 +113,6 @@ require (
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/otiai10/copy v1.14.0 // indirect
github.com/pelletier/go-toml/v2 v2.2.2 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/pkg/errors v0.9.1 // indirect
Expand Down Expand Up @@ -171,7 +169,6 @@ require (
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
k8s.io/metrics v0.31.1 // indirect
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 // indirect
oras.land/oras-go/v2 v2.5.0 // indirect
sigs.k8s.io/controller-runtime v0.18.2 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/kustomize/api v0.17.2 // indirect
Expand Down
8 changes: 0 additions & 8 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,6 @@ github.com/daviddengcn/go-colortext v1.0.0 h1:ANqDyC0ys6qCSvuEK7l3g5RaehL/Xck9EX
github.com/daviddengcn/go-colortext v1.0.0/go.mod h1:zDqEI5NVUop5QPpVJUxE9UO10hRnmkD5G4Pmri9+m4c=
github.com/defenseunicorns/go-oscal v0.6.0 h1:eflEKfk7edu4L4kWf6aNQpS94ljfGP8lgWpsPYNtE1Q=
github.com/defenseunicorns/go-oscal v0.6.0/go.mod h1:UHp2yK9ty2mYJDun7oNhbstCq6SAAwP4YGbw9n7uG6o=
github.com/defenseunicorns/pkg/helpers v1.1.3 h1:EVVuniq02qfAouR//AT0eoCngLWfFORj8H6+pI8M7uo=
github.com/defenseunicorns/pkg/helpers v1.1.3/go.mod h1:F4S5VZLDrlNWQKklzv4v9tFWjjZNhxJ1gT79j4XiLwk=
github.com/dgraph-io/badger/v3 v3.2103.5 h1:ylPa6qzbjYRQMU6jokoj4wzcaweHylt//CH0AKt0akg=
github.com/dgraph-io/badger/v3 v3.2103.5/go.mod h1:4MPiseMeDQ3FNCYwRbbcBOGJLf5jsE0PPFzRiKjtcdw=
github.com/dgraph-io/ristretto v0.1.1 h1:6CWw5tJNgpegArSHpNHJKldNeq03FQCwYvfMVWajOK8=
Expand Down Expand Up @@ -294,10 +292,6 @@ github.com/open-policy-agent/opa v0.68.0 h1:Jl3U2vXRjwk7JrHmS19U3HZO5qxQRinQbJ2e
github.com/open-policy-agent/opa v0.68.0/go.mod h1:5E5SvaPwTpwt2WM177I9Z3eT7qUpmOGjk1ZdHs+TZ4w=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/otiai10/copy v1.14.0 h1:dCI/t1iTdYGtkvCuBG2BgR6KZa83PTclw4U5n2wAllU=
github.com/otiai10/copy v1.14.0/go.mod h1:ECfuL02W+/FkTWZWgQqXPWZgW9oeKCSQ5qVfSc4qc4w=
github.com/otiai10/mint v1.5.1 h1:XaPLeE+9vGbuyEHem1JNk3bYc7KKqyI/na0/mLd/Kks=
github.com/otiai10/mint v1.5.1/go.mod h1:MJm72SBthJjz8qhefc4z1PYEieWmy8Bku7CjcAqyUSM=
github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM=
github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
github.com/peterbourgon/diskv v2.0.1+incompatible h1:UBdAOUP5p4RWqPBg048CAvpKN+vxiaj6gdUUzhl4XmI=
Expand Down Expand Up @@ -559,8 +553,6 @@ k8s.io/metrics v0.31.1 h1:h4I4dakgh/zKflWYAOQhwf0EXaqy8LxAIyE/GBvxqRc=
k8s.io/metrics v0.31.1/go.mod h1:JuH1S9tJiH9q1VCY0yzSCawi7kzNLsDzlWDJN4xR+iA=
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 h1:pUdcCO1Lk/tbT5ztQWOBi5HBgbBP1J8+AsQnQCKsi8A=
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
oras.land/oras-go/v2 v2.5.0 h1:o8Me9kLY74Vp5uw07QXPiitjsw7qNXi8Twd+19Zf02c=
oras.land/oras-go/v2 v2.5.0/go.mod h1:z4eisnLP530vwIOUOJeBIj0aGI0L1C3d53atvCBqZHg=
sigs.k8s.io/controller-runtime v0.18.2 h1:RqVW6Kpeaji67CY5nPEfRz6ZfFMk0lWQlNrLqlNpx+Q=
sigs.k8s.io/controller-runtime v0.18.2/go.mod h1:tuAt1+wbVsXIT8lPtk5RURxqAnq7xkpv2Mhttslg7Hw=
sigs.k8s.io/e2e-framework v0.4.0 h1:4yYmFDNNoTnazqmZJXQ6dlQF1vrnDbutmxlyvBpC5rY=
Expand Down
26 changes: 26 additions & 0 deletions src/cmd/common/common.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package common

import (
"fmt"
"strings"

"github.com/defenseunicorns/lula/src/internal/template"
)

func ParseTemplateOverrides(setFlags []string) (map[string]string, error) {
overrides := make(map[string]string)
for _, flag := range setFlags {
parts := strings.SplitN(flag, "=", 2)
if len(parts) != 2 {
return overrides, fmt.Errorf("invalid --set flag format, should be .root.key=value")
}

if !strings.HasPrefix(parts[0], "."+template.CONST+".") && !strings.HasPrefix(parts[0], "."+template.VAR+".") {
return overrides, fmt.Errorf("invalid --set flag format, path should start with .const or .var")
}

path, value := parts[0], parts[1]
overrides[path] = value
}
return overrides, nil
}
30 changes: 27 additions & 3 deletions src/cmd/common/viper.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,21 @@ package common

import (
"errors"
"fmt"
"os"
"strings"

"github.com/defenseunicorns/lula/src/internal/template"
"github.com/defenseunicorns/lula/src/pkg/message"
"github.com/spf13/viper"
)

const (
VLogLevel = "log_level"
VTarget = "target"
VSummary = "summary"
VLogLevel = "log_level"
VTarget = "target"
VSummary = "summary"
VConstants = "constants"
VVariables = "variables"
)

var (
Expand Down Expand Up @@ -70,6 +74,24 @@ func GetViper() *viper.Viper {
return v
}

// GetTemplateConfig loads the constants and variables from the viper config
func GetTemplateConfig() (map[string]interface{}, []template.VariableConfig, error) {
var constants map[string]interface{}
var variables []template.VariableConfig

err := v.UnmarshalKey(VConstants, &constants)
if err != nil {
return nil, nil, fmt.Errorf("unable to unmarshal constants into map: %v", err)
}

err = v.UnmarshalKey(VVariables, &variables)
if err != nil {
return nil, nil, fmt.Errorf("unable to unmarshal variables into slice: %v", err)
}

return constants, variables, nil
}

func isVersionCmd() bool {
args := os.Args
return len(args) > 1 && (args[1] == "version" || args[1] == "v")
Expand All @@ -78,6 +100,8 @@ func isVersionCmd() bool {
func setDefaults() {
v.SetDefault(VLogLevel, "info")
v.SetDefault(VSummary, false)
v.SetDefault(VConstants, make(map[string]interface{}))
v.SetDefault(VVariables, make([]interface{}, 0))
}

func printViperConfigUsed() {
Expand Down
Loading

0 comments on commit 5d1f232

Please sign in to comment.