Skip to content

Library to generate Kubernetes objects from a devfile and accessing the devfile registry

License

Notifications You must be signed in to change notification settings

devfile/library

Repository files navigation

Devfile Library

Go Apache2.0 License OpenSSF Best Practices OpenSSF Scorecard

About

The Devfile Parser library is a Golang module that:

  1. parses a devfile as specified by the api & schema.
  2. writes to the specified devfile with the updated data.
  3. generates Kubernetes objects for the various devfile resources.
  4. defines util functions for the devfile.
  5. downloads resources from a parent devfile if specified in the devfile.

Private repository support

Tokens are required to be set in the following cases:

  1. parsing a devfile from a private repository
  2. parsing a devfile containing a parent devfile from a private repository [1]
  3. parsing a devfile from a private repository containing a parent devfile from a public repository [2]

Set the token for the repository:

parser.ParserArgs{
	...
	// URL must point to a devfile.yaml
	URL: <url-to-devfile-on-supported-git-provider-repo>/devfile.yaml
	Token: <repo-personal-access-token>
	...
}

Note: The url must also be set with a supported git provider repo url.

Minimum token scope required:

  1. GitHub: Read access to code
  2. GitLab: Read repository
  3. Bitbucket: Read repository

Note: To select token scopes for GitHub, a fine-grained token is required.

For more information about personal access tokens:

  1. GitHub docs
  2. GitLab docs
  3. Bitbucket docs

[1] Currently, this works under the assumption that the token can authenticate the devfile and the parent devfile; both devfiles are in the same repository.

[2] In this scenario, the token will be used to authenticate the main devfile.

Usage

The function documentation can be accessed via pkg.go.dev.

  1. To parse a devfile, visit parse.go source file

    // ParserArgs is the struct to pass into parser functions which contains required info for parsing devfile.
    parserArgs := parser.ParserArgs{
     	Path:                           path,
     	FlattenedDevfile:               &flattenedDevfile,
     	ConvertKubernetesContentInUri:  &convertKubernetesContentInUri
     	RegistryURLs:                   registryURLs,
     	DefaultNamespace:               defaultNamespace,
     	Context:                        context,
     	K8sClient:                      client,
    		ExternalVariables:              externalVariables,
     }
    
    // Parses the devfile and validates the devfile data
    // if top-level variables are not substituted successfully, the warnings can be logged by parsing variableWarning
    devfile, variableWarning, err := devfilePkg.ParseDevfileAndValidate(parserArgs)
  2. To override the HTTP request and response timeouts for a devfile with a parent reference from a registry URL, specify the HTTPTimeout value in the parser arguments

       // specify the timeout in seconds  
       httpTimeout := 20 
       parserArgs := parser.ParserArgs{
          HTTPTimeout: &httpTimeout
       }
  3. To get specific content from devfile

    // To get all the components from the devfile
    components, err := devfile.Data.GetComponents(DevfileOptions{})
    
    // To get all the components from the devfile with attributes tagged - tool: console-import
    // & import: {strategy: Dockerfile}
    components, err := devfile.Data.GetComponents(DevfileOptions{
       Filter: map[string]interface{}{
     		"tool": "console-import",
     		"import": map[string]interface{}{
     			"strategy": "Dockerfile",
     		},
     	},
    })
    
    // To get all the volume components
    components, err := devfile.Data.GetComponents(DevfileOptions{
     	ComponentOptions: ComponentOptions{
     		ComponentType: v1.VolumeComponentType,
     	},
    })
    
    // To get all the exec commands that belong to the build group
    commands, err := devfile.Data.GetCommands(DevfileOptions{
     	CommandOptions: CommandOptions{
     		CommandType: v1.ExecCommandType,
     		CommandGroupKind: v1.BuildCommandGroupKind,
     	},
    })
  4. To get the Kubernetes objects from the devfile, visit generators.go source file

     // To get a slice of Kubernetes containers of type corev1.Container from the devfile component containers
     containers, err := generator.GetContainers(devfile)
    
     // To generate a Kubernetes deployment of type v1.Deployment
     deployParams := generator.DeploymentParams{
     	TypeMeta:          generator.GetTypeMeta(deploymentKind, deploymentAPIVersion),
     	ObjectMeta:        generator.GetObjectMeta(name, namespace, labels, annotations),
     	InitContainers:    initContainers,
     	Containers:        containers,
     	Volumes:           volumes,
     	PodSelectorLabels: labels,
     }
     deployment := generator.GetDeployment(deployParams)
  5. To update devfile content

    // To update an existing component in devfile object
    err := devfile.Data.UpdateComponent(v1.Component{
    	    Name: "component1",
    	    ComponentUnion: v1.ComponentUnion{
    	    	Container: &v1.ContainerComponent{
    	    		Container: v1.Container{
    	    			Image: "image1",
                 },
             },
         },
    })
    
    // To add a new component to devfile object
    err := devfile.Data.AddComponents([]v1.Component{
         {
            Name: "component2",
            ComponentUnion: v1.ComponentUnion{
                Container: &v1.ContainerComponent{
                    Container: v1.Container{
                        Image: "image2",
                    },
                },
            },
         },
    })
    
    // To delete a component from the devfile object
    err := devfile.Data.DeleteComponent(componentName)
  6. To write to a devfile, visit writer.go source file

    // If the devfile object has been created with devfile path already set, can simply call WriteYamlDevfile to writes the devfile
    err := devfile.WriteYamlDevfile()
    
    // To write to a devfile from scratch
    // create a new DevfileData with a specific devfile version
    devfileData, err := data.NewDevfileData(devfileVersion)
    
    // set schema version
    devfileData.SetSchemaVersion(devfileVersion)
    
    // add devfile content use library APIs
    devfileData.AddComponents([]v1.Component{...})
    devfileData.AddCommands([]v1.Commands{...})
    ......
    
    // create a new DevfileCtx
    ctx := devfileCtx.NewDevfileCtx(devfilePath)
    err = ctx.SetAbsPath()
    
    // create devfile object with the new DevfileCtx and DevfileData
    devfile := parser.DevfileObj{
     	Ctx:  ctx,
     	Data: devfileData,
    }
    
    // write to the devfile on disk
    err = devfile.WriteYamlDevfile()
  7. To parse the outerloop Kubernetes/OpenShift component's uri or inline content, call the read and parse functions

    // Read the YAML content
    values, err := ReadKubernetesYaml(src, fs)
    
    // Get the Kubernetes resources
    resources, err := ParseKubernetesYaml(values)
  8. By default, the parser will set all unset boolean properties to their spec defined default values. Clients can override this behaviour by specifiying the parser argument SetBooleanDefaults to false

    setDefaults := false
    parserArgs := parser.ParserArgs{
     	SetBooleanDefaults:               &setDefaults,
    }
  9. When parsing a devfile that contains a parent reference, if the parent uri is a supported git provider repo url with the correct personal access token, all resources from the parent git repo excluding the parent devfile.yaml will be downloaded to the location of the devfile being parsed. Note: The URL must point to a devfile.yaml

    schemaVersion: 2.2.0
    ...
    parent:
       uri: <uri-to-parent-devfile>/devfile.yaml
    ...
  10. By default, the library downloads the Git repository resources associated with the Git URL that is mentioned in a devfile uri field. To turn off the download, pass in the DownloadGitResources property in the parser argument

downloadGitResources := false
parserArgs := parser.ParserArgs{
 	DownloadGitResources:               &downloadGitResources,
}
  1. To download/access files from a private repository like a private GitHub use the Token property
parserArgs := parser.ParserArgs{
 	Token: "my-PAT",
}
src: YamlSrc{
 	URL: "http://github.com/my-private-repo",
 	Token: "my-PAT",
}
values, err := ReadKubernetesYaml(src, fs, nil)

If you would like to use the mock implementation for the DevfileUtils interface method defined in pkg/devfile/parser/util/interface.go, then use

var devfileUtilsClient DevfileUtils
devfileUtilsClient = NewMockDevfileUtilsClient()
devfileUtilsClient.DownloadInMemory(params)

Projects using devfile/library

The following projects are consuming this library as a Golang dependency

Tests

To run unit tests and api tests. Visit library tests to find out more information on tests

make test

Issues

Issues are tracked in the devfile/api repo with the label area/library

Releases

The devfile/library releases are created annually or on demand. For existing devfile/library releases, please check the release page.

Create a New Release

The steps to create a new release are:

  • Create a separate branch for the particular release, for example, v2.0.x

  • Run this script to generate release changelog.

# generate a changelog for all the changes since release v2.0.0
./scripts/changelog-script.sh v2.0.0
  • Create a new release here with a new tag (having the same name with the above branch - e.g. v2.0.x) and copy the generated changelog to the details

Security

Further information for the devfile organization security policy can be found here.

Contributing

Please see our contributing.md.

About

Library to generate Kubernetes objects from a devfile and accessing the devfile registry

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages