Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AutoArchiveLogs option to send all workflow pod logs to Artifact Repository #1791

Closed
wants to merge 2 commits into from

Conversation

irlevesque
Copy link

Fixes: #1790

@CLAassistant
Copy link

CLAassistant commented Nov 22, 2019

CLA assistant check
All committers have signed the CLA.

@@ -83,6 +83,8 @@ type KubeConfig struct {
type ArtifactRepository struct {
// ArchiveLogs enables log archiving
ArchiveLogs *bool `json:"archiveLogs,omitempty"`
// AutoArchiveLogs enables log archiving for all tasks regardless of workflow configuration
AutoArchiveLogs *bool `json:"autoArchiveLogs,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe ForceArchiveLogs is more explicit?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about DefaultArchiveLogs? This change wouldn't force logs to be archived (users explicitly setting s3/artifactory destinations in template wouldn't be covered by this).

@@ -834,6 +834,9 @@ func (woc *wfOperationCtx) addArchiveLocation(pod *apiv1.Pod, tmpl *wfv1.Templat
break
}
}
if woc.artifactRepository.AutoArchiveLogs != nil && *woc.artifactRepository.AutoArchiveLogs {
needLocation = true
}
Copy link
Member

@simster7 simster7 Nov 26, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, currently it's not clear that even when enabling AutoArchiveLogs the user will also need to enable ArchiveLogs for the logs to be archived. Maybe a couple of lines below should actually be:

	tmpl.ArchiveLocation = &wfv1.ArtifactLocation{
		ArchiveLogs: woc.artifactRepository.ArchiveLogs || woc.artifactRepository.AutoArchiveLogs,
	}

Or something similar

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on clarity. I think your proposal sounds reasonable.

@simster7
Copy link
Member

simster7 commented Dec 3, 2019

On more thought, I'm unsure if a config in ArtifactRepository should be allowed to change the behavior of Workflows downstream (as opposed to only changing its own behavior). It seems to me that the Repository shouldn't be able to "force" Workflows that use it to do anything. In the case of ArchiveLogs the Repository simply "allows" log archiving – which is only affecting its own behavior.

What do you think? Can you think of another way to do this that would make the Workflow the active player in this?

I want to know what @jessesuen thinks of this too

@sarabala1979 sarabala1979 added this to the v2.5 milestone Dec 4, 2019
@alexec
Copy link
Contributor

alexec commented Dec 16, 2019

I've just managed to re-implement this almost exactly the same in #1860

@alexec
Copy link
Contributor

alexec commented Dec 16, 2019

@irlevesque do you want to compare the two solutions, they're not exactly the same, as I re-use the ArchiveLogs flag, as this already work if there are any outputs.

@irlevesque
Copy link
Author

Hi @alexec, my rationale for using a new variable was to avoid a change of behavior for existing users of the ArchiveLogs feature, who may have vested interest in maintaining the current functionality (and may indeed not want this new functionality).

@irlevesque irlevesque closed this Dec 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Globally-defined logging to artifact repository
5 participants