-
Notifications
You must be signed in to change notification settings - Fork 243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Support for Devfile V2 #3216
Conversation
57bfe93
to
0578186
Compare
/retest |
@adisky Can you also verify with the devfiles in https://github.com/elsony/devfile2-registry? I tried with them, and the following devfiles failed:
All due to the same issue:
It seems when |
I'm not sure that this is how it should work. |
@kadel If there's one project set in the devfile, we decided to sync to Wouldn't we need to do the same for Edit: Or perhaps we need to rethink how we handle syncing when one, or multiple projects (or starterProjects) are defined in a devfile and handle in a separate issue? |
@adisky brought up on slack that As such, the rest of the expected functionality of |
@johnmcollier We can open an issue now also, what about the devfiles in v2 registry are you going to update them until |
@adisky I'll check with Elson to see if we can update them to add/use the |
Sure, also I have logged separate issue to add devfile v2 registry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a couple preliminary comments.
Can you also add more integration test cases to test the V2 devfiles?
pkg/devfile/adapters/common/utils.go
Outdated
return component.Type == common.DevfileComponentTypeDockerimage | ||
// Currently odo only uses devfile components of type container, since most of the Che registry devfiles use it | ||
if component.Container != nil { | ||
klog.V(3).Infof("Found component \"%v\" with name \"%v\"\n", common.ContainerComponentType, component.Container.Name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the log level of this be 4 instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious as to how much does this change affect devfile v1? Since we're removing the DevfileComponentTypeDockerimage
check..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After parsing V1, DevfileComponentTypeDockerimage
is converted to type Container
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@johnmcollier Log level is same is before.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please change to 4, may be good to do it in this PR 👍
} | ||
|
||
// component must be specified | ||
if action.Component == nil || *action.Component == "" { | ||
return fmt.Errorf("Actions must reference a component") | ||
if command.Exec.Component == "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if we should handle this as part of the devfile validator, rather than here. Then it would save a bunch of time if the user was using a devfile with invalid commands
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When is the other devfile validator invoked? Because if it is validated before push, a user can change the devfile and push would have to work with probably an invalid devfile. 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@johnmcollier I also think validation should happen after parsing, but refactoring validation logic is not part of this PR :)
you can raise new issue for that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@adisky Fair enough 😄
} | ||
|
||
// must specify a command | ||
if action.Command == nil || *action.Command == "" { | ||
return fmt.Errorf("Actions must have a command") | ||
if command.Exec.CommandLine == "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as before, wonder if we should move this into the devfile validator.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
replied above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thx for making these changes. I posted some comments during my initial review of this PR.
|
||
// Get the supported actions | ||
supportedCommandActions, err := getSupportedCommandActions(data, command) | ||
command = updateGroupforCustomCommand(commandName, groupType, command) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this can be inside the if block if commandName != "" {
and you wouldnt need to do the check inside the function updateGroupforCustomCommand()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, group for custom commands has to be inserted before validateCommand
, otherwise it would fail with group not found
. So it cannot bet inside if commandName != ""
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is the custom command update func call here for devfile v1?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On second thought, I think we should remove this function altogether.
I dont see a point of updating a command's group since its not mandatory. Check my other comment on validateCommand
func
In addition to thinking we dont need it here, looking at this logic; we are updating commands which do not match and validating here; which may not break anything functionally but i think its not right. For example; a user may bring a devfile with many commands which do not have to adhere to odo criteria i.e.; they do not need a group. But as we're looping to search for a command match, we update such devfile commands.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@maysunfaisal we are updating group for commands specified by odo flag --build-command
, --run-command
, not for all commands. updateGroupforCustomCommand
, we can remove the validation for group but group needs to be inserted for custom command, this is internally only so that user does not need to specify it in devfile.
type Attributes map[string]string | ||
// CommandGroupType describes the kind of command group. | ||
// +kubebuilder:validation:Enum=build;run;test;debug | ||
type DevfileCommandGroupType string |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be DevfileCommandGroupKind
instead?
since the devfile command group has kind:
- exec:
id: download dependencies
commandLine: "npm install"
component: runtime
workingDir: ${CHE_PROJECTS_ROOT}/nodejs-web-app/app
group:
kind: build
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is done to be in sync from here https://github.com/devfile/kubernetes-api/blob/master/pkg/apis/workspaces/v1alpha1/commands.go#L20, In future we might utilize these package directly.
} | ||
|
||
// component must be specified | ||
if action.Component == nil || *action.Component == "" { | ||
return fmt.Errorf("Actions must reference a component") | ||
if command.Exec.Component == "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When is the other devfile validator invoked? Because if it is validated before push, a user can change the devfile and push would have to work with probably an invalid devfile. 🤔
Exec *Exec `json:"exec,omitempty"` | ||
|
||
// Type of workspace command | ||
Type DevfileCommandType `json:"type,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DevfileCommand has property Type of DevfileCommandType
which can be Exec; but DevfileCommand also has a property Exec of type Exec
? I think this can be cleaned up, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have not added it, it is from schema
https://github.com/devfile/kubernetes-api/blob/master/pkg/apis/workspaces/v1alpha1/commands.go#L67
Actually for simplicity we have removed other types of command, so DevfileCommandType
signifies which type of command it contains.
but valid point though, as currently we have only exec
so it makes no sense here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
alright, thx for the explanation. Do we need the Type
, maybe it can be cleaned up later?
@@ -305,81 +305,64 @@ func (a Adapter) waitAndGetComponentPod(hideSpinner bool) (*corev1.Pod, error) { | |||
|
|||
// Executes all the commands from the devfile in order: init and build - which are both optional, and a compulsary run. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you pls fix existing typo compulsory* or change to mandatory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
// Only add runinit to the expected commands if the component doesn't already exist | ||
// This would be the case when first running the container |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think these comments can be updated to reflect your command map
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
err = exec.ExecuteDevfileRunAction(&a.Client, action, command.Name, compInfo, show) | ||
} | ||
} | ||
// Get Init Command |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a few questions here with which you might help
- Do we want to have three separate if blocks and execute the commands in order; or do we want to loop the map and use the previous idea of having a predefined order and execute the commands?
- What can we do about refactoring code between Kubernetes Adapter and Docker Adapter execDevfile() since they're pretty much the same. See my comment here Devfile Adapter Cleanup/Refactor #3015 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a few questions here with which you might help
- Do we want to have three separate if blocks and execute the commands in order; or do we want to loop the map and use the previous idea of having a predefined order and execute the commands?
Predefined order is not required as we have kind
associated with each command, for now lets leave it like this, It has to be updated with events
support.
- What can we do about refactoring code between Kubernetes Adapter and Docker Adapter execDevfile() since they're pretty much the same. See my comment here #3015 (comment)
Agreed, that can be refactored.
@@ -344,83 +344,61 @@ func getPortMap(context string, endpoints []versionsCommon.DockerimageEndpoint, | |||
|
|||
// Executes all the commands from the devfile in order: init and build - which are both optional, and a compulsary run. | |||
// Init only runs once when the component is created. | |||
func (a Adapter) execDevfile(pushDevfileCommands []versionsCommon.DevfileCommand, componentExists, show bool, containers []types.Container) (err error) { | |||
func (a Adapter) execDevfile(commandsMap common.PushCommandsMap, componentExists, show bool, containers []types.Container) (err error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pls see my comment on execDevfile() for the Kubernetes Adapter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
replied above.
} | ||
|
||
// Description of the projects, containing names and sources locations | ||
// CommandsItems |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please update the descriptions for all the exported items in this file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
@johnmcollier regarding integration tests, as i know @mik-dass discussed with @kadel, We would not write additional tests for |
/retest |
1 similar comment
/retest |
Codecov Report
@@ Coverage Diff @@
## master #3216 +/- ##
==========================================
- Coverage 45.58% 45.50% -0.08%
==========================================
Files 110 111 +1
Lines 10953 10939 -14
==========================================
- Hits 4993 4978 -15
+ Misses 5481 5477 -4
- Partials 479 484 +5
Continue to review full report at Codecov.
|
I've done some basic tests with both devfile v1 and v2, haven't found any big issues so far. /approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kadel The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Add devfiles v2 examples for springboot and nodejs
fix regression caused by rebase to master. Also add github, zip as supported project types.
dde62b0
to
bf8fed0
Compare
fix log levels to v4 fix error formatting add case no's in test cases update some comments
Remove validation for group
Id: c.Name, | ||
WorkingDir: action.Workdir, | ||
// Env: | ||
// Label: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't we remove these?
ErrorNoComponents = "no components present" | ||
ErrorNoDockerImageComponent = fmt.Sprintf("odo requires atleast one component of type '%s' in devfile", common.DevfileComponentTypeDockerimage) | ||
ErrorNoComponents = "no components present" | ||
ErrorNoContainerComponent = fmt.Sprintf("odo requires atleast one component of type '%s' in devfile", common.ContainerComponentType) | ||
) | ||
|
||
// ValidateComponents validates all the devfile components |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we make this function lowercase? it seems to not be used anywhere outside this package.
apiVersion, okApi := r["apiVersion"] | ||
|
||
// Get "schemaVersion" value from map for devfile V2 | ||
schemaVersion, okSchema := r["schemaVersion"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what happens if both are provided, apiVersion and schemaVersion?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is no field apiVersion
for V2 and no field schemaVersion
for V1, so they cannot be provided together, json validation would fail for such case.
func convertV1EndpointsToCommon(e DockerimageEndpoint) common.Endpoint { | ||
return common.Endpoint{ | ||
// Attributes: | ||
// Configuration: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we remove these commented attributes?
small nitpicks, rest the code is great 👍 |
@adisky the |
/lgtm |
agreed it is not needed, it was in my mind too, I would log a refactoring issue tomorrow. |
I still don't like how we have a folder named 1.0.0 which we have 2.0.0 features implemented.. At the moment, I don't mind merging this as well once tests pass. But we will 100% need to refactor this later on so that it's less confusing on implementation. /lgtm |
/retest Please review the full test history for this PR and help us cut down flakes. |
@cdrage Please read the doc about v2 implementation plan, Hope it makes things clear to you!! |
/retest Please review the full test history for this PR and help us cut down flakes. |
1 similar comment
/retest Please review the full test history for this PR and help us cut down flakes. |
* add mik-dass and dharmit as top level approvers (redhat-developer#3189) * Add .gitignore files for jekyll / site (redhat-developer#3171) **What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` /kind code-refactoring **What does does this PR do / why we need it**: Updates the `.gitignore` for html and jekyll information. As I kept having to `git clean` the dir. **Which issue(s) this PR fixes**: N/A **How to test changes / Special notes to the reviewer**: N/A Signed-off-by: Charlie Drage <charlie@charliedrage.com> * Performance improvement for odo catalog list components when using devfiles (redhat-developer#3112) * feat: parallel fetching of component types to improve performance Fix redhat-developer#3105 as we now also fetch devfiles for completion, which was previously prohibitively slow. * fix: add missing mutex, rename existing one for clarity * refactor: simplify getDevfileWith by computing link within function * docs: add documentation, update existing one * refactor: rename wg to devfileWG for clarity * fix: improper test setup * fix: only filter components if there's no error * fix: re-implement using channels, handle error and timeout better * fix: no need to pass error * fix: populate base in GetDevfileIndex, use registry URL, hiding path * fix: properly retrieve all devfiles Performance is not that great so investigating better options. * fix: use new ConcurrentTasks support, allowing proper error reporting * fix: adapt to dynamic registries * feat: simplify ConcurrentTask by hiding WaitGroup It was too easy to forget calling Done in ToRun leading to deadlocks. * fix: broken tests * fix: wording [skip ci] * fix: remove unused function, added doc [skip ci] * fix: remove unused ErrorWrapper [skip ci] * fix: rename package import [skip ci] * fix: test getDevfileIndexEntries directly, remove now unused function * JSON output for service list in experimental mode (redhat-developer#3186) * JSON output for service list in experimental mode * Changes to tests as per PR feedback redhat-developer#3186 (review) * Adds a document for debugging applications using devfiles (redhat-developer#3187) * Adds a document for debugging applications using devfiles * Updated syntax and improving the wording Signed-off-by: mik-dass <mrinald7@gmail.com> * Actually wait for the project to be deleted.. (redhat-developer#2397) Actually wait.. Signed-off-by: Charlie Drage <charlie@charliedrage.com> * Run devfile docker url tests without kube config (redhat-developer#3223) * Run devfile docker url tests without kube config Signed-off-by: John Collier <John.J.Collier@ibm.com> * Update target Signed-off-by: John Collier <John.J.Collier@ibm.com> * Fix typo Signed-off-by: John Collier <John.J.Collier@ibm.com> * Setting golang version uniformly accross repo and updating docs (redhat-developer#3204) * Setting golang version uniformly accross repo and updating docs - All golang version references now uniformly updated to 1.12 - Adding all golang version references to development.adoc Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Adding golang warning to admonition block Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Removing unnessasary space Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Adding back warning as github does not render admonition in normal view Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Fixing the warning to look better Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Fixing up the list title Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Fixing up admonishments Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Fixing commits as per @cdrage comments Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Fix odo describe not listing all the ports after first url is created (redhat-developer#3224) * Fix odo describe not listing all the ports after first url is created This PR fixes redhat-developer#3201. Signed-off-by: Denis Golovin dgolovin@redhat.com * update tests Co-authored-by: Tomas Kral <tkral@redhat.com> * Support listing devfile components with no devBuild command (redhat-developer#3172) * Don't require devfiles to have a devBuild command Signed-off-by: John Collier <John.J.Collier@ibm.com> * Update integration tests Signed-off-by: John Collier <John.J.Collier@ibm.com> * Update integration test Signed-off-by: John Collier <John.J.Collier@ibm.com> * Update integration test for odo create Signed-off-by: John Collier <John.J.Collier@ibm.com> * Make devRun check case-insensitive Signed-off-by: John Collier <John.J.Collier@ibm.com> * replace secured with secure (redhat-developer#3238) A typo existing in sample file * Running devfile integration tests on kubernetes cluster (redhat-developer#3233) * Document to run tests on kubernetes cluster * Addressed review comment * Updated the doc as per reviewer request * Modify operator hub tests to run in separate namespaces (redhat-developer#3239) * Modify operator hub tests to run in separate namespaces **What type of PR is this?** /kind cleanup /kind failing-test **What does does this PR do / why we need it**: This PR ensures that each test spec for operator hub is run in a separate namespace. This is needed because these tests were written in a stringent fashion which tied the hands while writing additional tests for the feature. This PR should help take the tests out of the "odo test statistics" dashboard. **Which issue(s) this PR fixes**: Fixes redhat-developer#3193 **How to test changes / Special notes to the reviewer**: CI should pass. Specially for Operator Hub tests under `tests/integration/operatorhub` * Changes as per PR review * Remove createEtcdClusterService function and more separate specs - Removed the function because it *seemed* to be leading to race conditions - Separating specs for the same reason * Loop for two operators * Make operator hub tests sequential We're doing this because parallel run is causing numerous failures in CI and local. In spite of debugging it for some time, we're not able to fix it. However, we're on a deadline because operator hub failures are showing up across the PRs. Hence, for time being, we're making these tests sequential. redhat-developer#3244 * Add devfile command flags to watch (redhat-developer#3075) * save non default devfile cmd in env.yaml for push and watch Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Add and update tests for envinfo Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Update watch test to incl new watch param Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * watch integration test Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Display msg when devfile commands are changed via odo push Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Change Logic for watch flag + feedback 1 Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Update Watch tests with feedback Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * disable watch test on kube since we now use oc Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Watch test feedback 2 Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Adding sparseCheckoutDir functionality to devfiles (redhat-developer#3042) * Initial commit * added devfile * added tests * commented out tests * change to validpath statement * changed method of extracting zip * changed pathsToUnzip to pathToUnzip * Added error message when no files are unzipped * cleaned up conditional prefix trim * Changes from feedback * feedback changes and unit tests for util func * check for empty pathToUnzip * changed error message format * cleaned up path separator for windows * fixed return pathToUnzip * use hassuffix * moved to function * fromslash * minor fixes * lowercase fix for test * Run test locally against 4.* Cluster (redhat-developer#3218) * Run test locally against 4.* Cluster * Added review comments * Added few more review comments * Addressed review comments * Addressed review comments * Addressed correct format review comments * Update samples for deploying a devfile in deploying-a-devfile-using-odo.adoc (redhat-developer#3163) * Update devfile doc samples [skip ci] Signed-off-by: John Collier <John.J.Collier@ibm.com> * Address review comments Signed-off-by: John Collier <John.J.Collier@ibm.com> * Fix formatting Signed-off-by: John Collier <John.J.Collier@ibm.com> * Replace multiple expect statement with MatchAllInOutput and DontMatchAllInOutput (redhat-developer#3251) * Replaced multiple expect assert statement with MatchAllInOutput * Replaced multiple expect check within same variable with MatchAllInOutput and DontMatchAllInOutput throughout the test * Formatted string array which were exceeding 120 character * add state to ingress list and describe (redhat-developer#3160) * add state to ingress list and describe Signed-off-by: Stephanie <stephanie.cao@ibm.com> * use URL struct in url describe and list * add unit tests Signed-off-by: Stephanie <stephanie.cao@ibm.com> * modify integration test and add more unit tests Signed-off-by: Stephanie <stephanie.cao@ibm.com> * remove types created * address review comment * delete removed tests Signed-off-by: Stephanie <stephanie.cao@ibm.com> * return error if the error is not notfound Signed-off-by: Stephanie <stephanie.cao@ibm.com> * list multiple ulrs in test Signed-off-by: Stephanie <stephanie.cao@ibm.com> * Don't prompt for namespace if pushtarget is Docker (redhat-developer#3248) Signed-off-by: John Collier <John.J.Collier@ibm.com> * Fix windows incompatibility with /tmp (redhat-developer#3252) * Fix checkForImageStream to return correct result. (redhat-developer#3281) * Add Support for Devfile V2 (redhat-developer#3216) * Add Devfile Parser V2, Update Common Structs (redhat-developer#3188) * Add Devfile Parser for Version 2.0.0 Added Devfile V2 Go Structures Added converter for v2 ro common types Signed-off-by: adisky <adsharma@redhat.com> * Add example V2 devfile added example nodejs V2 devfile * Add Common Types as V2 Add common types as v2 Add Converter from common to v1 * Updated example devfile with group * Fixes command.go and kubernetes adapter (redhat-developer#3194) Signed-off-by: mik-dass <mrinald7@gmail.com> * Fixes utils of k8s adapter (redhat-developer#3195) * Update Command Logic to use groups (redhat-developer#3196) * Updates create logic to v2 (redhat-developer#3200) * Fixes utils of docker docker adapter (redhat-developer#3198) Signed-off-by: mik-dass <mrinald7@gmail.com> * Update Docker and Kubernetes adapter to use groups (redhat-developer#3206) * Update Docker and Kubernetes adapter to use groups * Update Some Validation Incorporate some review comments for commands Map Update Some Validation logic * Updated Logs for V2 (redhat-developer#3208) Fixed interface implementation for v2 Updated logs refactored commands.go * Avoid String Pointers (redhat-developer#3209) While converting v1 to v2 types, string pointers are prone to cause null pointer error. This PR updates struct fields from string pointers to string * Update commands check * Fixes lower and upper case for commands (redhat-developer#3219) * Fixes type of project and components for devfile v1 (redhat-developer#3228) * Update testing utils (redhat-developer#3220) * Update command tests Updated Command tests to v2 Removed some cases like command type validation, that will be validated by schema only. * Fix common adapters tests All devfile.adapters.common unit tests are fixed * Add tests for Mismatched type Fix devfile.adapters.Kubernetes.Component unit tests * Add TestCase for default command * Design proposal: Event notification support for build and application status for IDE integration for devfile scenarios (redhat-developer#2550) (redhat-developer#3177) * Add event notification proposal [skip ci] * Update event-notification.md * Update event-notification.md * Update event-notification.md * Update event-notification.md * Update event-notification.md * Update event-notification.md * Update event-notification.md * Update event-notification.md * Fix Integration tests for devfile v2 (redhat-developer#3236) * Fix Integration tests Correct volume structure Fix supervisord binary insertion for docker Insert command and args in build container fr docker * Fix Integration tests Revert commands, args removal Add commands, args in v2 common structs Fix duplicate volume error * Fixes unit tests (redhat-developer#3240) Signed-off-by: mik-dass <mrinald7@gmail.com> * Add devfiles V2 examples (redhat-developer#3253) Add devfiles v2 examples for springboot and nodejs * Fix regression by sparse checkout dir PR (redhat-developer#3258) fix regression caused by rebase to master. Also add github, zip as supported project types. * Address review comments (redhat-developer#3267) * Address review comments part 2 fix log levels to v4 fix error formatting add case no's in test cases update some comments * Address review comments part 2 Remove validation for group Co-authored-by: Mrinal Das <mrinald7@gmail.com> Co-authored-by: Jonathan West <jgwest@users.noreply.github.com> * devfile push Integration test on kubernetes cluster (redhat-developer#3041) * Running devfile push integration test on kubernetes cluster * Updated xenial insecure registry path * Modifying Runner variable name to cliRunner * Common project Create and Delete function according to the running cluster * Added checks for CI to copy kubeconfig to temp dir * Skipping Kubeconfig set to temporary config file for prow * Set Kubeconfig in temporary config file on kubernetes cluster only * Fixes Kubeconfig code sync * Separate out function for getting cliRunner * Release 1.2.2 of odo (redhat-developer#3294) **What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` /kind feature **What does does this PR do / why we need it**: Releases 1.2.1 of odo **Which issue(s) this PR fixes**: N/A **How to test changes / Special notes to the reviewer**: N/A Signed-off-by: Charlie Drage <charlie@charliedrage.com> * Fixes list and describe of secure URLs for not pushed components (redhat-developer#3269) * Don't show imagestreams error on experimental mode (redhat-developer#3265) * Don't show imagestreams error on experimental mode Signed-off-by: John Collier <John.J.Collier@ibm.com> * Remove unnecessary line Signed-off-by: John Collier <John.J.Collier@ibm.com> * Move where error message printed Signed-off-by: John Collier <John.J.Collier@ibm.com> * Use %q instead Signed-off-by: John Collier <John.J.Collier@ibm.com> * remove openshift logo from readme (redhat-developer#3301) * Remove unnecessary preference set (redhat-developer#3287) * report error can't find the app (redhat-developer#2927) * Update build/VERSION to 1.2.2 (redhat-developer#3306) **What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` /kind feature **What does does this PR do / why we need it**: Updates the build version to 1.2.2 **Which issue(s) this PR fixes**: N/A **How to test changes / Special notes to the reviewer**: N/A Signed-off-by: Charlie Drage <charlie@charliedrage.com> * Update tests to use java-openliberty (redhat-developer#3318) * Removing install script and update other references (redhat-developer#3202) * Removing installer scripts and references in other scripts and readme Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Removing installer reference from development.adoc Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Adding a message to installer script so as to not break it for existing users Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Removing unused variable which is no longer needed Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Removing installer again Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Removing unused secret directory (redhat-developer#3290) * devfile delete Integration test on kubernetes cluster (redhat-developer#2913) * Run devfile delete integration test on kubernetes cluster * Adding insecure registry in Docker configuration xenial ubuntu * Fix namespace issue for url create --now (redhat-developer#3321) * Add --devfile flag support (redhat-developer#3118) * feat: add --devfile flag support Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * fix: fix gosec G307 issue Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * fix: update error handling Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * style: Update help and error messages Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * fix: Improve performance of error handling Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * style: Improve error message Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * fix: Fix remove file error on Windows Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * fix: Use existing structure to handle temp file Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * refactor: improve code readability and structure Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * docs: update variable comments Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * feat: --devfile value supports http(s) Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * feat: remove --devfile for all other commands Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * feat: add new design for existing devfile support Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: update TestCopyFile test cases Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: refactor file copy util function for test Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: fix TestCopyFile test cases Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: add integration tests for existing devfile Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * fix: gosec issue Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: fix docker url integration tests Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * fix: context support Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: fix docker url integration tests Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * fix: handle edge cases and add related tests Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * refactor: separate spinner for existing devfile Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: fix docker watch tests Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: fix url tests Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * Updating travis env variable and adding docker version info (redhat-developer#3068) * 'odo login' command should suggest 'odo project list' instead of 'odo projects' (redhat-developer#3262) * use a more specific error message for when image is not found (redhat-developer#3322) * use a specific error message for when image is not found * added amit's changes * devfile create Interation test on kubernetes cluster using travis CI (redhat-developer#2884) * Run devfile create Interation test on kubernetes cluster * Skipping s2i image related test on kubernetes cluster * Updating url delete to apply `--now` on devfile (redhat-developer#3139) * Creating a function for setting devfile path * updating url delete to do devfile push post delete Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Adding URL now flag delete test Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Fixing missing `-f` Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Changing position of copyexample Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Merging url delete with now with create Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Removing old devfilepath logic Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * Adding Devfile flag to url delete Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * CompleteDevfilePath now happens irrespective of now Signed-off-by: Mohammed Zeeshan Ahmed <mohammed.zee1000@gmail.com> * list devfile components catalog in json format (redhat-developer#3264) * list catalog in json format for devfile components Signed-off-by: Stephanie <stephanie.cao@ibm.com> * list devfile registries in list * combined two list into one single list Signed-off-by: Stephanie <stephanie.cao@ibm.com> * add omitempty * Fix unsafe data race in ExecuteCommand (redhat-developer#3310) **What type of PR is this?** /kind bug **What does does this PR do / why we need it**: Fixes a potential flake with regards to when writing to cmdOutput at the same time that log.ErrorF(...) is reading from it. **Which issue(s) this PR fixes**: Closes redhat-developer#2828 **How to test changes / Special notes to the reviewer**: N/A, honestly not too sure * Fix Docker Supervisord Vol Initialization (redhat-developer#3129) * apply stashed supervisord docker changes from PR Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * rebase 1 with master Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Converts v1 devfiles to v2 (redhat-developer#3279) * Converts v1 devfiles to v2 * Moves init tests to devfilesV1 folder * Adding oc to bin path for multistage testing (redhat-developer#3351) * Revert "Adding oc to bin path for multistage testing (redhat-developer#3351)" (redhat-developer#3360) This reverts commit fa9f7b9. * resolved the gosec issue failing unit tests (redhat-developer#3358) * resolved the gosec issue failing unit tests * resolved typo in the comment * DRYed the code * Warn if --app and --all-apps coexist (redhat-developer#2941) * url list routes and ingress (redhat-developer#3305) * list routes and ingress Signed-off-by: Stephanie <stephanie.cao@ibm.com> * add unit tests * add integration tests and describe partially works Signed-off-by: Stephanie <stephanie.cao@ibm.com> * finished unit test and integration for describe urls * address review comments Signed-off-by: Stephanie <stephanie.cao@ibm.com> * add more unit tests Signed-off-by: Stephanie <stephanie.cao@ibm.com> * fix unit test failure Signed-off-by: Stephanie <stephanie.cao@ibm.com> * fix integration test failure Signed-off-by: Stephanie <stephanie.cao@ibm.com> * use ownerreference to distinguish real route vs route created from ingress Signed-off-by: Stephanie <stephanie.cao@ibm.com> * fix integration test Signed-off-by: Stephanie <stephanie.cao@ibm.com> * address review comment Signed-off-by: Stephanie <stephanie.cao@ibm.com> * fix a bug Signed-off-by: Stephanie <stephanie.cao@ibm.com> * fix a bug Signed-off-by: Stephanie <stephanie.cao@ibm.com> * also test secure state in url describe and list Signed-off-by: Stephanie <stephanie.cao@ibm.com> * try to avoid G404 error in unit test Signed-off-by: Stephanie <stephanie.cao@ibm.com> * Fix 'odo version' integration test. (redhat-developer#3326) * Validate application exist before describe it (redhat-developer#3349) * Update events (redhat-developer#3370) **What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` /kind documentation [skip ci] **What does does this PR do / why we need it**: Updates the events to list that they happened in the past **Which issue(s) this PR fixes**: N/A **How to test changes / Special notes to the reviewer**: N/A Signed-off-by: Charlie Drage <charlie@charliedrage.com> * Updates the v2 schema to the latest (redhat-developer#3352) * fix: properly check that odo version contains the server URL (redhat-developer#3365) * fix: properly check that odo version contains the server URL Fixes redhat-developer#3342 * fix: simplify test by removing server url regexp check * Create periodic script for odo periodic job (redhat-developer#3376) * Adds debug command for devfiles (redhat-developer#3059) * Adds debug command for devfiles * Fixes comments and variable names * Adds devfile debug test on kubernetes cluster * Fixes adapter code for debug Signed-off-by: mik-dass <mrinald7@gmail.com> * Converts v1 debug test devfile to v2 * Moves container arg and command overriding to a new function * Simplifies conditions in test * Modifies execDevfile for kubernetes component adapter * Simplies error checking logic in TestUpdateContainersWithSupervisord * Removes unnecessary else if conditions * Installs socat before running tests on kubernetes cluster * Show error message if no registries are configured (redhat-developer#3339) * Show error message if no registries are configured Signed-off-by: John Collier <John.J.Collier@ibm.com> * Clean up Signed-off-by: John Collier <John.J.Collier@ibm.com> * Added few supported container images to e2e image test (redhat-developer#3315) * Fix multiple registry related issues (redhat-developer#3341) * feat: fix registry issues Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * docs: improve error/help message Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: improve registry tests Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * feat: github raw url conversion support Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: refactor testing output format Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * test: fix breaking tests Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * refactor: add validation for raw url conversion Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * Update dependencies in glide.yaml for make to succeed in building odo binary (redhat-developer#3369) * Execute `glide update -v` to update the dependencies * Use ContainerJSON in place of Container struct * Push support: Event notification support for build and application status for IDE integration for devfile scenarios (redhat-developer#3003) * Add machine readable event logging support * Add machine readable event logging support * Add machine readable event logging support * Add machine readable event logging support * Add machine readable event logging support * Add machine readable event logging support * Add machine readable event logging support * Remove --devfile * Remove incorrect constraint * Add odo push -o json example * Rebase and add debug action logging to new methods * Adding oc binary to bin path for multi stage test infra (redhat-developer#3362) * Adding oc binary to bin path for multi stage test infra * Added file permission locally and excluding it from docker file * Created different dockerfile to avoid master break and proceed with multistage changes * Update devfile log levels from 3 to 4 to be consistent (redhat-developer#3408) Signed-off-by: Maysun J Faisal <maysun.j.faisal@ibm.com> * Design proposal: secure registry support (redhat-developer#3329) * docs: add secure registry design proposal Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * docs: add create component section Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * Add command design for basic auth Signed-off-by: jingfu wang <jingfu.j.wang@ibm.com> * Proposal for odo deploy (redhat-developer#3368) * docs: Initial design proposal for odo deploy command Initial design proposal for adding a odo deply command. This is related to redhat-developer#3300 Signed-off by: Neeraj Laad <neeraj.laad@gmail.com> * Updated future evolution Updated future evolution Signed-off-by: Neeraj Laad <neeraj.laad@gmail.com> * Updates to design Signed-off-by: Neeraj Laad <neeraj.laad@gmail.com> * Update odo-deploy.md * Updated metadata to attributes Updated metadata to attributes * Fixed formatting Fixed formatting for yaml * [skip ci] updated design doc * [skip ci] made credentials flag optional made credentials flag optional * [skip ci] clarify the deplyment example clarify the deplyment example * [skip ci] Updated proposal to sue buildconfig / kaniko * [skip ci] update attribites to metadata * [skip ci] add `alpha.` suffix to metadata keys to indicate this is temporary * [skip ci] Use COMPONENT_NAME as placeholder * Updating script path in Dockerfile for multistage test (redhat-developer#3412) * Invalid ANSI colouring on Windows when doing devfile push (redhat-developer#3327) * show catalog component list -o json as before if experimental mode is off (redhat-developer#3399) Signed-off-by: Stephanie <stephanie.cao@ibm.com> * removed deprecated and nodejs 8 images from e2e tests and README (redhat-developer#3414) * removed deprecated and nodejs 8 images from e2e tests * removed the unsupported images from readme and removed bucharest-gold/centos7-s2i-node from everywhere as the repo has been archived * adjusted the table * Update changelog script (redhat-developer#3421) **What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` /kind code-refactoring **What does does this PR do / why we need it**: Updates the changelog script since we were having issues with changelog not completing due to hitting the GitHub API limit. **Which issue(s) this PR fixes**: N/A **How to test changes / Special notes to the reviewer**: N/A Signed-off-by: Charlie Drage <charlie@charliedrage.com> * Release 1.2.3 (redhat-developer#3422) **What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` /kind feature **What does does this PR do / why we need it**: Release 1.2.3 of odo **Which issue(s) this PR fixes**: N/A **How to test changes / Special notes to the reviewer**: N/A Signed-off-by: Charlie Drage <charlie@charliedrage.com> * fix 'Timeout out' typo and fix formatting (redhat-developer#3404) * fix 'Timeout out' typo and fix formating * fix typo * pull changes from upstream master * add wwhrd check execption for knative.dev/pkg/test/... Co-authored-by: Tomas Kral <tkral@redhat.com> Co-authored-by: Charlie Drage <charlie@charliedrage.com> Co-authored-by: Chris Laprun <metacosm@users.noreply.github.com> Co-authored-by: Dharmit Shah <shahdharmit@gmail.com> Co-authored-by: Mrinal Das <mrinald7@gmail.com> Co-authored-by: John Collier <John.J.Collier@ibm.com> Co-authored-by: Mohammed Ahmed <mohammed.zee1000@gmail.com> Co-authored-by: Denis Golovin <dgolovin@users.noreply.github.com> Co-authored-by: ji chen <jichenjc@cn.ibm.com> Co-authored-by: Priti Kumari <pkumari@redhat.com> Co-authored-by: Maysun J Faisal <31771087+maysunfaisal@users.noreply.github.com> Co-authored-by: CameronMcWilliam <Cameron.McWilliam@ibm.com> Co-authored-by: Amit Rout <arout@redhat.com> Co-authored-by: Stephanie Cao <stephanie.cao@ibm.com> Co-authored-by: Zheng Xiao Mei <xmzheng@cn.ibm.com> Co-authored-by: Aditi Sharma <adsharma@redhat.com> Co-authored-by: Jonathan West <jgwest@users.noreply.github.com> Co-authored-by: Jingfu Wang <jingfu.j.wang@ibm.com> Co-authored-by: Devang Gaur <dgaur@redhat.com> Co-authored-by: Girish Ramnani <gramnani@redhat.com> Co-authored-by: Girish Ramnani <girishramnani95@gmail.com> Co-authored-by: Neeraj Laad <neeraj.laad@gmail.com>
What type of PR is this?
/kind feature
What does does this PR do / why we need it:
Detailed Information on implementation can be found in this doc
https://docs.google.com/document/d/11BVhP1gViu5C3IGYQpiulX1lUkSGOG8p-lOJq4I5mUs/edit#heading=h.bw3vmyuddut1
Which issue(s) this PR fixes:
Fixes #3145
Fixes #3140
#2939 #2938
How to test changes / Special notes to the reviewer:
Build binaries can be downloaded from unit prow job - https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_odo/3216/pull-ci-openshift-odo-master-unit/4685/artifacts/unit/dist/bin/