Add a new method to the PipelineResource interface for customization …

[dlorenc]

…of the Output directory

Changes

We previously initialized each Output Resource with an empty directory. This changes the
behavior to initialize this with a custom set of files per resource. The behavior now is
the same as before, with each resource embedding a helper to create empty directories.

Over time we can customize this where necessary.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

• Includes tests (if functionality changed/added)
• Includes docs (if user facing)
• Commit messages follow commit message best practices

Ref #1214

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dlorenc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [dlorenc]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[bobcatfish]

As you know from #1188 I'm not a huge fan of struct embedding XD I have a couple other ideas but I'm okay with going ahead with this too! vs. #1188 I like that EmptyDirOutputResource has a clear name and responsibility.

This is more of a long term thing and I think you were suggesting something like this @dlorenc , but
Looking at the fact that we've got GetUploadSteps and GetOutputInitializationSteps now, maybe we can split up the input/output functionality?

And we could do it without struct embedding, dig this partial example:

// Interface for PipelineResources
type PipelineResourceInterface interface {
       // These are all the same
	GetName() string
	GetType() PipelineResourceType
	Replacements() map[string]string

        // The functionality we've been looking at is now input vs. output and `PodModifier` is the interface
        GetInputStepModifer() PodModifier
        GetOutputStepModifier() PodModifier
}

// This interface returns steps to add
type ResourceSteps interface {
     Get(sourcePath string) ([]Step, error)
}

// This interface returns volumes to add
type ResourceVolumes interface {
    Get(spec *TaskSpec) ([]corev1.Volume, error)
}

// Describes how to modify a Pod to implement an input or output resource 
type PodModifier interface {
       // For steps that have to go before the "actual steps" (for outputs too!) copied from the proposal in #238 :innocent: 
	GetBeforeSteps(name, sourcePath string) ResourceSteps
        // For steps that go after the "actual steps"
  	GetAfterSteps(name, sourcePath string) ResourceSteps
        // I am scared of the fact that a TaskSpec is getting passed in but that's another story :D
        GetVolumes(spec *TaskSpec) ResourceVolumes
}

// Used when an empty dir needs to be created
type CreateEmptyDirSteps struct {}
func (CreateEmptyDirSteps*) Get(name, sourcePath string) ([]Step, error) {
    return []Step{CreateDirStep(name, sourcePath)}, nil
}

// The pod modifier for image inputs
type ImageResourceInputPodModifier struct {}
func (ImageResourceInputPodModifier *) GetBeforeSteps(name, sourcePath string) ResourceSteps {
    return CreateEmptyDirSteps{}
}
// ....

// NoResourceSteps just returns no steps
// Can be used by `NoPodModifications` but also can be returned by a PodModifier for any function that returns `ResourceSteps`
type NoResourceSteps {}
func (NoResourceSteps*) Get(sourcePath string) ([]Step, error) { return []Step{}, nil }

// NoResourceVolumes just returns no volumes
type NoResourceVolumes {}
func (NoResourceVolumes*) Get(spec *TaskSpec) ([]corev1.Volume, error) { return []corev1.Volume{}, nil}

// A PodModifier that does nothing
// For example GetOutputStepModifier could return this if outputs arent supported
type NoPodModifications struct { }

// Here is how PodModifier would indicate nothing needs to be done
func (NoPodModifications*) GetBeforeSteps(name, sourcePath string) ResourceSteps { return NoResourceSteps{} }
func (NoPodModifications*) GetAfterSteps(name, sourcePath string) ResourceSteps { return NoResourceSteps{} }
func (NoPodModifications*) GetVolumes(spec *TaskSpec)  ResourceVolumes { return NoResourceVolumes{} }

What I like about this is that most of the code that has to be written is going to be describing the functionality of the PipelineResources only, using (hopefully) small, reusable, clearly named and easily testable chunks of logic.

It's interesting also b/c whatever we settle on for PipelineResourceInterface is pretty much gonna turn into the interface folks have to implement for extensible resource (#238 ) .

But after all that I'm happy to lgtm this as well and keep the discussion going!

[dlorenc]

I like your proposal around the "GetInputModifier" and "GetOutputModifier"s! I think we'll have to consider the order in which they're applied, but I think it's pretty good overall and might actually make testing much easier.

Let me give that one a try. I had a similar idea, but I was also trying to separate out Input and Output resources at the same time and that got too complicated.

[afrittoli]

@dlorenc are you going to submit a new version of this? Shall we talk about this as part of the PipelineResources WG as well?

[dlorenc]

Closing this for now.