-
Notifications
You must be signed in to change notification settings - Fork 26
feat: Implement first phase of move2kube workflow #375
feat: Implement first phase of move2kube workflow #375
Conversation
83b6325
to
0ccc391
Compare
0ccc391
to
ae2e4e7
Compare
prebuilt-tasks/src/main/java/com/redhat/parodos/tasks/git/GitCloneTask.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/main/java/com/redhat/parodos/tasks/git/GitCommitTask.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/test/java/com/redhat/parodos/tasks/git/GitBranchTaskTest.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/test/java/com/redhat/parodos/tasks/git/GitBranchTaskTest.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/test/java/com/redhat/parodos/tasks/git/GitBranchTaskTest.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/test/java/com/redhat/parodos/tasks/git/GitBranchTaskTest.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/test/java/com/redhat/parodos/tasks/git/GitCloneTaskTest.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/test/java/com/redhat/parodos/tasks/git/GitCloneTaskTest.java
Outdated
Show resolved
Hide resolved
...ow-examples/src/main/java/com/redhat/parodos/examples/move2kube/task/Move2KubeTransform.java
Outdated
Show resolved
Hide resolved
...ow-examples/src/main/java/com/redhat/parodos/examples/move2kube/task/Move2KubeTransform.java
Outdated
Show resolved
Hide resolved
...ow-examples/src/main/java/com/redhat/parodos/examples/move2kube/task/Move2KubeTransform.java
Outdated
Show resolved
Hide resolved
...ow-examples/src/main/java/com/redhat/parodos/examples/move2kube/task/Move2KubeTransform.java
Outdated
Show resolved
Hide resolved
...ow-examples/src/main/java/com/redhat/parodos/examples/move2kube/task/Move2KubeTransform.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/main/java/com/redhat/parodos/tasks/git/GitBranchTask.java
Show resolved
Hide resolved
|
||
@Slf4j | ||
@AllArgsConstructor | ||
public class GitCommitTask extends BaseWorkFlowTask { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see opportunity to share the code with GitBranchTask
. There is good amount of the same code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
because the getRepo you mean? It's overwritten after in the workflow examples. the other code I cannot see a room to share the code with
} | ||
|
||
private String getMove2KubeAPIEndpoint() { | ||
return "http://localhost:8081/api/v1"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should make it configurable
} | ||
|
||
@Bean(name = "transformWorkFlowChecker") | ||
@Checker(cronExpression = "*/5 * * * * ?") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How much time transformation is taking? Do you think checking it so often make sense?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a human process, so can take a lot, but for demo purposes is great.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this can be defined only by code? meaning, changing the corn-expression requires releasing a new version of the workflow?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, is the only way.
workflow-examples/src/main/java/com/redhat/parodos/examples/move2kube/task/Move2KubeBase.java
Outdated
Show resolved
Hide resolved
workflow-examples/src/main/java/com/redhat/parodos/examples/move2kube/task/Move2KubePlan.java
Outdated
Show resolved
Hide resolved
planApi.startPlanning(workspaceID, projectID); | ||
|
||
for (int i = 1; i <= 10; ++i) { | ||
GetPlan200Response plan = planApi.getPlan(workspaceID, projectID); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we want to wait exponentially here to get the plan? We could try using async instead. Please take a look how @gciavarrini implemented api wait.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The thing is that it's not waiting at all. it's returning 404 does not exists.. so we need to wait until the plan is created
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was exactly the reason why Gloria implemented it in this way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ohh, got it. The main problem that I can see, I need to implement the async way for move2kube API for an example workflow, does that make sense?
workflow-examples/src/main/java/com/redhat/parodos/examples/move2kube/task/Move2KubePlan.java
Outdated
Show resolved
Hide resolved
"You need to complete some information for your transformation in the following url <a href=\"%s\"> %s</a>", | ||
url, url); | ||
|
||
// @TODO userID is the ID, but we need the username, so hardcode it here for now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please create a task to track this change
a4cae90
to
94b1235
Compare
somehow the workflow-examples' test is still reported in coverage, shall we remove it and only runs it in maven build? |
94b1235
to
95193fd
Compare
prebuilt-tasks/src/main/java/com/redhat/parodos/tasks/git/GitArchiveTask.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/main/java/com/redhat/parodos/tasks/git/GitCommitTask.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/main/java/com/redhat/parodos/tasks/git/GitUtils.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/main/java/com/redhat/parodos/tasks/git/GitArchiveTask.java
Outdated
Show resolved
Hide resolved
prebuilt-tasks/src/main/java/com/redhat/parodos/tasks/git/GitBranchTask.java
Outdated
Show resolved
Hide resolved
|
||
private static File createSampleZip() throws IOException { | ||
String zipFileName = "/tmp/test%s.zip".formatted(UUID.randomUUID()); | ||
String tempDir = "/tmp/%s/".formatted(UUID.randomUUID().toString()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is an API for that: java.nio.file.Files.createTempFile
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know, and I do not need to create the file at all, only the path for creating a sample ZIP:
private static File createSampleZip() throws IOException {
String zipFileName = "/tmp/test%s.zip".formatted(UUID.randomUUID());
String tempDir = "/tmp/%s/".formatted(UUID.randomUUID().toString());
File zipFile = new File(zipFileName);
String[] fileNames = { "output/project-name/deploy/bar.txt", "output/project-name/deploy/foo.txt",
"output/project-name/scripts/bar.sh", "output/project-name/scripts/foo.sh" };
byte[] buffer = new byte[1024];
FileOutputStream fos = new FileOutputStream(zipFile);
ZipOutputStream zos = new ZipOutputStream(fos);
for (String fileName : fileNames) {
File file = new File("%s%s".formatted(tempDir, fileName));
if (file.getParentFile() != null && !file.getParentFile().exists()) {
assertTrue(file.getParentFile().mkdirs());
}
assertTrue(file.createNewFile());
ZipEntry zipEntry = new ZipEntry(fileName);
zos.putNextEntry(zipEntry);
FileInputStream fis = new FileInputStream(file);
int length;
while ((length = fis.read(buffer)) > 0) {
zos.write(buffer, 0, length);
}
fis.close();
zos.closeEntry();
}
zos.close();
return zipFile;
}
WorkReport report = task.execute(workContext); | ||
|
||
// then | ||
assertNull(report.getError()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- pls prefer the use of assertThat.
- The order of parameters counts as it used to generate the error message:
assertXYX(expected, actual)
...low-examples/src/test/java/com/redhat/parodos/examples/move2kube/task/Move2KubeTaskTest.java
Outdated
Show resolved
Hide resolved
|
||
private static String move2KubeProjectIDCtxKey = "move2KubeProjectID"; | ||
|
||
PlanApi planApi; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
private?
|
||
PlanApi planApi; | ||
|
||
ProjectOutputsApi projectOutputsApi; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
private?
95193fd
to
0c41e89
Compare
This commit adds the first phase of the move2kube workflow example. A user give any git repo, and it'll will be transformed using move2kube, two new folders will be commited (deploy and scripts) inside the repository where the transformed application can be used. This PR introduces a new 3party dependency that it's the openapi client from move2kube. The workflow is the following one: ``` sequenceDiagram participant User participant Parodos participant API as Move2Kube API participant web as Move2kube Web participant k8s as OpenshiftCluster rect rgb(221, 255, 221) User ->> Parodos: Enter params Note over User,Parodos: Parameters needed:<br/>- Git Repo<br/>- Git Credentials<br/>- GitTargetBranch<br/>- Profile as workspaceID? end rect rgb(221, 255, 221) Parodos ->> API: Get Workspace API ->> Parodos: return workspace ID Parodos ->> API: Post Project (Workflow ID as project ID) API ->> Parodos: Return project ID end Note over Parodos,API: Here Parodos prepare the plan rect rgb(221, 255, 221) Parodos --> Parodos: Git Clone Parodos --> Parodos: Git archive end rect rgb(221, 255, 221) Parodos --> API: Push ZIP file as project input1 POST /workspaces/{workspace-id}/projects/{project-id}/inputs Parodos ->> API: POST /workspaces/{workspace-id}/projects/{project-id}/plan Parodos ->> API: Get /workspaces/{workspace-id}/projects/{project-id}/plan Note over API,User: Here user get notified to start transformation Parodos ->> User: Link to project-id move2kube page User ->> web: User execute the transformation step in move2kube loop Every minute Parodos ->> API: Is transformation ready? end Note over API,User: Trasnformation completed successfully Parodos ->> API: Get outputs Parodos ->> Parodos: Branch code Parodos ->> Parodos: Commit changes end Parodos ->> Parodos: Push code to the specified branch Note over Parodos: Openshift deployment Parodos ->> Parodos: Build images for project Parodos ->> k8s: Apply all manifests Parodos ->> User: Workflow completed successfully with route ``` All the green boxes are created, the not-highlight ones are the ones for the next PR. For move2kube an workspace need to be created, and the config that I used is the following one: ``` move2kube: containerruntime: docker minreplicas: "2" services: demo: "8080": servicetype: Ingress urlpath: / enable: true port: "8080" target: default: clustertype: Kubernetes ingress: host: localhost ingressclassname: "" tls: "" imageregistry: quay.io: logintype: no authentication namespace: parodos url: quay.com transformers: types: - ArgoCD - Buildconfig - ClusterSelector - ComposeAnalyser - ComposeGenerator - ContainerImagesPushScriptGenerator - DockerfileDetector - DockerfileImageBuildScript - DockerfileParser - DotNetCore-Dockerfile - Gradle - Jar - Jboss - Kubernetes - KubernetesVersionChanger - Liberty - Maven - Parameterizer - ReadMeGenerator - Tomcat - WarAnalyser - WarRouter - WinWebApp-Dockerfile - ZuulAnalyser transformerselector: "" ``` To run move2kube the easiest way that I find is the following command: ``` docker run --rm -it -p 8081:8080 quay.io/konveyor/move2kube-ui ``` Fix: https://issues.redhat.com/browse/FLPATH-356 Partial-Fix: https://issues.redhat.com/browse/FLPATH-355 Demo: https://drive.google.com/file/d/1k9RwRI4PPBwinDWRyARvTsbFGiQD6vjh/view?usp=sharing Signed-off-by: Eloy Coto <eloy.coto@acalustra.com>
Signed-off-by: Eloy Coto <eloy.coto@acalustra.com>
5c6410e
to
7a36e95
Compare
70% of coverage already 🎉 The only thing missing is the getRepo in the GitTask. I would like to keep like This is something that we need to revisit. Are you happy if I create a new task Almost all the assertTrue,assertEquals are now in assertThat; maybe one is missing, but I didn't spot it. Can I merge this and work on the async of the client on a different PR? That will unlock the git push prebuild-task |
Signed-off-by: Eloy Coto <eloy.coto@acalustra.com>
7a36e95
to
92630f7
Compare
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: RichardW98 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/lgtm |
This commit adds the first phase of the move2kube workflow example. A user give any git repo, and it'll will be transformed using move2kube, two new folders will be commited (deploy and scripts) inside the repository where the transformed application can be used.
This PR introduces a new 3party dependency that it's the openapi client from move2kube.
The workflow is the following one:
All the green boxes are created, the not-highlight ones are the ones for the next PR.
For move2kube an workspace need to be created, and the config that I used is the following one:
To run move2kube the easiest way that I find is the following command:
Fix: https://issues.redhat.com/browse/FLPATH-356
Partial-Fix: https://issues.redhat.com/browse/FLPATH-355
Demo: https://drive.google.com/file/d/1k9RwRI4PPBwinDWRyARvTsbFGiQD6vjh/view?usp=sharing