🌱 Add Pathfinder assessment migration to CLI

[aufi]

Extending tackle CLI tool with option to export Pathfinder assessments and import it to Konveyor API assessments endpoint.

$ ./tackle migrate-assessments

More information about CLI usage: https://github.com/konveyor/tackle2-hub/tree/main/hack/tool#readme

Related to #489

[aufi]

@jmontleon @jortel Thinking about usage in operator upgrade:

• option 1 - Does it make sense first dump Pathfinder data on Konveyor 0.2 version, then upgrade Konveyor and import assessments to 0.3 as part of the upgrade steps (option 1) or
• option 2 - upgrade Konveyor, but keep running Pathfinder, then run the assessment migration live directly from Pathfinder (without Hub proxy) to Hub API without creating temp files and kill Pathfinder after migration succeeded?

[jmontleon]

I'm trying to think about this from a workflow where the operator is orchestrating it. Option 2 sounds cleaner to me, from a logic standpoint. Is it possible to run the migration utility process within a pod (maybe a job would be appropriate) that can do the migration between pathfinder and hub?

[aufi]

A job sounds good to me too to execute the python migration script (@dymurray, was it planned to have the script part of the hub image or a special one?). It'd need to get endpoints for Pathfinder (probably the internal one) and Hub API to run the migration.

[jortel]

Perhaps I am being confused by the terminology. When I read execute the python migration script I feel like we are rolling something special for this.
I think the original plan is the simplest:

• Add functionality to the export/import tool to work with Assessments. This is what the tool is for.
• Add support for hub API version 3 (for all things including assessment).
• Package the tool with the operator.
• The operator uses to tool to export/import assessments.

This is an application level migration that does should be performed/orchestrated by the operator, not the hub.

The workflow:

1. The operator uses the tool to export assessments though the hub API (this is how the tool works).
2. upgrade tackle.
3. import the assessments
4. On success, Get rid of pathfinder.

If the operator doing this in a job is better, it seems to just add moving parts but open to understanding why it's better.

[aufi]

Previous comments included also the "execution" part of the assessments migration that will be part of operator repo, so that was misleading, sorry for mixing it.

One more open question is about credentials/authorization - the tool expects username&password which is not known to the Hub (AFAIK), just Keycloak knows it. But as @mansam suggested, an addon token could be used for this assessments migration purpose. would this addon token (that is known to the operator too) be suitable for calling Hub API by the tool instead of login/password @jortel?

[jortel]

The hub has a builtin auth provider the is used for addon tokens.
Yes, we have the operator build/sign an auth token with assessment scopes and signed using the key in the secret. I will prototype that.

[aufi]

If there is an assessment on the application, import is skipped, so the script can run multiple times without failing.

[aufi]

The export part is commented out until we decide best matching scenario for execution with operator upgrade. Examples from import part:

$ ./tackle migrate-assessments -d ./tackle-data-010223
Getting auth token via Hub from http://192.168.39.5
Starting Pathfinder Assessments to Konveyor Assessment migration.
Processing Pathfinder assessments..
# Assessment for Application 2
Category 1 Application details
  Question 1 Does the application development team understand and actively develop the application?
    Answer 5 greenfield application
  Question 2 How is the application supported in production?
    Answer 5 DevOps approach with the same team building the application and supporting it in production
  Question 3 How much time passes from when code is committed until the application is deployed to production?
    Answer 5 1-7 days
  Question 4 How often is the application deployed to production?
    Answer 4 Weekly
  Question 5 What is the application's mean time to recover (MTTR) from failure in a production environment?
    Answer 5 Less than 1 hour
  Question 6 Does the application have legal and/or licensing requirements?
    Answer 4 None
  Question 7 Which model best describes the application architecture?
    Answer 5 Independently deployable components
Category 2 Application dependencies
  Question 1 Does the application require specific hardware?
    Answer 4 Requires CPU that is supported by red Hat
  Question 2 What operating system does the application require?
    Answer 5 Standard Linux distribution
  Question 3 Does the vendor provide support for a third-party component running in a container?
    Answer 6 No third-party components required
  Question 4 Incoming/northbound dependencies
    Answer 3 Many dependencies exist, can be changed because the systems are internally managed
  Question 5 Outgoing/southbound dependencies
    Answer 5 No outgoing/southbound dependencies
Category 3 Application architecture
  Question 1 How resilient is the application? How well does it recover from outages and restarts?
    Answer 4 Application employs resilient architecture patterns (examples: circuit breakers, retry mechanisms)
  Question 2 How does the external world communicate with the application?
    Answer 5 HTTP/HTTPS
  Question 3 How does the application manage its internal state?
    Answer 5 Stateless or ephemeral container storage
  Question 4 How does the application handle service discovery?
    Answer 4 Uses Kubernetes DNS name resolution
  Question 5 How is the application clustering managed?
    Answer 4 No cluster management required
Category 4 Application observability
  Question 1 How does the application use logging and how are the logs accessed?
    Answer 6 Logs are configurable (example: can be sent to stdout)
  Question 2 Does the application provide metrics?
    Answer 4 Metrics exposed using a third-party solution (examples: Dynatrace, AppDynamics)
  Question 3 How easy is it to determine the application's health and readiness to handle traffic?
    Answer 4 Dedicated, independent liveness and readiness endpoints
  Question 4 What best describes the application's runtime characteristics?
    Answer 4 Intermittent traffic with predictable CPU and memory usage
  Question 5 How long does it take the application to be ready to handle traffic?
    Answer 5 Less than 10 seconds
Category 5 Application cross-cutting concerns
  Question 1 How is the application tested?
    Answer 4 Highly repeatable automated testing (examples: unit, integration, smoke tests) before deploying to production; modern test practices are followed
  Question 2 How is the application configured?
    Answer 6 Configuration loaded from files in a single configurable location; environment variables used
  Question 4 How is the application deployed?
    Answer 6 Fully automated (GitOps), blue-green, or canary deployment
  Question 5 Where is the application deployed?
    Answer 6 Hybrid cloud (public and private cloud providers)
  Question 6 How mature is the containerization process, if any?
    Answer 4 Proficient with containers and container platforms (examples: Swarm, Kubernetes)
  Question 3 How does the application acquire security keys or certificates?
    Answer 7 Not required
PUT filled Assessment.
Done. 1 new Assessment(s) for Applications were migrated!

$ ./tackle migrate-assessments -d ./tackle-data-010223
Getting auth token via Hub from http://192.168.39.5
Starting Pathfinder Assessments to Konveyor Assessment migration.
Processing Pathfinder assessments..
# Assessment for Application 2
  Assessment already exists, skipping.
Done. Assessments for 0 Application(s) were migrated!

[mansam]

This all looks great.

[aufi]

Updated assessment migration part to do pathfinder query and migration to Hub API without temporary files. Also added --token option to provide auth token to skip login with username/password, that might not be known. Ready for review.

[mansam]

LGTM. I recommend rebasing on main to clear the go vet errors.