Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for Resource creation where template has same parameter templating #1283

Merged
merged 11 commits into from
Mar 27, 2019
42 changes: 41 additions & 1 deletion examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ For a complete description of the Argo workflow spec, please refer to https://gi
- [Kubernetes Resources](#kubernetes-resources)
- [Docker-in-Docker Using Sidecars](#docker-in-docker-aka-dind-using-sidecars)
- [Continuous Integration Example](#continuous-integration-example)

- [Custom Template Variable Referrence](#Custom Template Variable Referrence)
## Argo CLI

In case you want to follow along with this walkthrough, here's a quick overview of the most useful argo command line interface (CLI) commands.
Expand Down Expand Up @@ -1258,6 +1258,46 @@ spec:
mirrorVolumeMounts: true
```

## Custom Template Variable Referrence
In this example, we can see how we can use the other template language variable reference (E.g: Jinja) in Argo workflow template.
Argo will validate and resolve only the variable that starts with Argo allowed prefix
{***"item", "steps", "inputs", "outputs", "workflow", "tasks"***}

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: custom-template-variable-
spec:
entrypoint: hello-hello-hello

templates:
- name: hello-hello-hello
steps:
- - name: hello1
template: whalesay
arguments:
parameters: [{name: message, value: "hello1"}]
- - name: hello2a
template: whalesay
arguments:
parameters: [{name: message, value: "hello2a"}]
- name: hello2b
template: whalesay
arguments:
parameters: [{name: message, value: "hello2b"}]

- name: whalesay
inputs:
parameters:
- name: message
container:
image: docker/whalesay
command: [cowsay]
args: ["{{user.username}}"]

```

## Continuous Integration Example

Continuous integration is a popular application for workflows. Currently, Argo does not provide event triggers for automatically kicking off your CI jobs, but we plan to do so in the near future. Until then, you can easily write a cron job that checks for new commits and kicks off the needed workflow, or use your existing Jenkins server to kick off the workflow.
Expand Down
32 changes: 32 additions & 0 deletions test/e2e/functional/custom_template_variable.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# This template demonstrates the customer variable suppport.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: custom-template-variable-
spec:
entrypoint: hello-hello-hello

templates:
- name: hello-hello-hello
steps:
- - name: hello1
template: whalesay
arguments:
parameters: [{name: message, value: "hello1"}]
- - name: hello2a
template: whalesay
arguments:
parameters: [{name: message, value: "hello2a"}]
- name: hello2b
template: whalesay
arguments:
parameters: [{name: message, value: "hello2b"}]

- name: whalesay
inputs:
parameters:
- name: message
container:
image: docker/whalesay
command: [cowsay]
args: ["{{custom.variable}}"]
3 changes: 3 additions & 0 deletions workflow/common/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,9 @@ const (
KubeConfigDefaultVolumeName = "kubeconfig"
)

// GlobalVarWorkflowRootTags is a list of root tags in workflow which could be used for variable reference
var GlobalVarValidWorkflowVariablePrefix = []string{"item.", "steps.", "inputs.", "pod.", "workflow.", "tasks."}

// ExecutionControl contains execution control parameters for executor to decide how to execute the container
type ExecutionControl struct {
// Deadline is a max timestamp in which an executor can run the container before terminating it
Expand Down
15 changes: 15 additions & 0 deletions workflow/validate/validate.go
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,11 @@ func resolveAllVariables(scope map[string]interface{}, tmplStr string) error {
fstTmpl := fasttemplate.New(tmplStr, "{{", "}}")

fstTmpl.ExecuteFuncString(func(w io.Writer, tag string) (int, error) {

// Skip the custom variable references
if !checkValidWorkflowVariablePrefix(tag) {
return 0, nil
}
_, ok := scope[tag]
if !ok && unresolvedErr == nil {
if (tag == "item" || strings.HasPrefix(tag, "item.")) && allowAllItemRefs {
Expand All @@ -245,6 +250,16 @@ func resolveAllVariables(scope map[string]interface{}, tmplStr string) error {
return unresolvedErr
}

// checkValidWorkflowVariablePrefix is a helper methood check variable starts workflow root elements
func checkValidWorkflowVariablePrefix(tag string) bool {
for _, rootTag := range common.GlobalVarValidWorkflowVariablePrefix {
if strings.HasPrefix(tag, rootTag) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also add . to HasPrefix (e.g. rootTag + '.') so that things like stepsfoo won't match but steps. will match?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch. Thanks. I will fix it

return true
}
}
return false
}

func validateNonLeaf(tmpl *wfv1.Template) error {
if tmpl.ActiveDeadlineSeconds != nil {
return errors.Errorf(errors.CodeBadRequest, "templates.%s.activeDeadlineSeconds is only valid for leaf templates", tmpl.Name)
Expand Down
20 changes: 20 additions & 0 deletions workflow/validate/validate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1142,3 +1142,23 @@ func TestSpecBadSequenceCountAndEnd(t *testing.T) {
err := ValidateWorkflow(wf, true)
assert.Error(t, err)
}

var customVariableInput = `
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: hello-world-
spec:
entrypoint: whalesay
templates:
- name: whalesay
container:
image: docker/whalesay:{{user.username}}
`

// TestCustomTemplatVariable verifies custom template variable
func TestCustomTemplatVariable(t *testing.T) {
wf := unmarshalWf(customVariableInput)
err := ValidateWorkflow(wf, true)
assert.Equal(t, err, nil)
}