Design full Project Creation CLI Flow

[amandakys]

Parent Issue: #2388

Description

Design the full revamped Project Creation CLI flow which includes:

• Add-ons selection (Add "add-ons" flow to kedro new CLI command #2850)
• Template selection (e.g. pandas, pyspark, viz)
• Example selection

Context

• Introducing Utility Modules to Kedro  #2388 Outlines the idea of the revamped CLI Flow
• Create new spaceflight starters #2838 Describes the new starters that will be created that can be selected as templates and examples during the project creation flow.
• Add "add-ons" flow to kedro new CLI command #2850 Contains the design for the add-ons selection

[yetudada]

Another requirement from @NeroOkwa: "Making sure Kedro-Viz is discoverable at CLI creation".

[datajoely]

Some nice libraries for interactive command line interfaces:

• InquirePy
• Textual

[yetudada]

Design considerations

We will work on the assumption that the user will be able to create a project template in the following ways:

• Using the project creation CLI wizard
• Using the project creation CLI flags
• Using their custom starters

These assumptions mean that we should:

• Rework what happens when a user runs kedro new because the project creation CLI wizard starts running
• Create new CLI flags to match the project creation CLI wizard
• Maintain the current CLI flags that allow users to choose a starter, namely --starter=<path-to-starter>, --starter=name_of_starter and --checkout=version_number

What's out of scope for V1?

• We are not including Airflow and Databricks in this current flow; we will look at this after 0.19.0
• Perhaps later, we can deprecate the CLI command group, kedro starter list, if we assume that users will not want to see our list, but this remains to be verified
• We will not create a single home in our documentation for all tools; currently PySpark, linting, testing, Kedro-Viz are all in different places; look at this after 0.19.0 and add the link to the CLI

What does the project creation wizard look like?

The user should:

• Run kedro new
• Be asked to provide a project name
• Be asked if they want example code, i.e. spaceflights
• Be asked for project add-ons (or tools)

Note: I've called add-ons, tools. But this can be changed back.

(my-virtual-environment) ➜  kedro new 

Project name
===========
Please enter a human-readable name for your new project.
Spaces, hyphens, and underscores are allowed.
To skip this step in future use the `--name` CLI flag

 [New Kedro project]: Customer segmentation 


Project tools
===========
Here, you can select which tools you'd like to include. 
To read more about these tools and what they do, visit: docs.kedro.org
To skip this step in future use the `--tools` CLI flag

Tools 
1)  Linting  :  Adds linting with Ruff and Black
2)  Testing  :  Adds testing support with Pytest
3)  Logging  :  Adds more logging options
4)  Documentation  :   Adds documentation support with Sphinx
5)  Data structure  :  Creates a directory structure for storing data 
6)  Kedro-Viz  :  Adds setup for Kedro's native visualisation tool
7)  PySpark  :  Adds support for PySpark

Which add-ons would you like to include in your project? [None/1-4/all/1,3]: 1,2,7


Example code
============
Select whether you would like an example spaceflights pipeline included in your project.
To read more about this example, visit: docs.kedro.org
To skip this step in future use the `--example` CLI flag

Would you like to include an example [y/N]: y

What do the project creation CLI flags look like?

We need to create new ones for the project name (--name) and tools (--tools) and including example code (--example_code).

So it could be something like:

kedro new --name="Customer segmentation" --tools=lint,test,pyspark --example_code=y

Note: We need shorthand names for --tools, i.e. linting is lint. We also have yet to decide the default behaviour if a user does not indicate one or more flags. We should assume people would use this flow in the CLI, so putting them through the CLI wizard to get the response should not happen.

Questions for our users

What happens if you don't provide a CLI flag, for:

• Project name?
• Example code?
• Add-ons?

For example: If you ran kedro new --name="Customer segmentation" what do you expect would happen?

Additional questions for us

How do you enter default when there are multiple options, e.g. the add-ons workflow?

[amandakys]

Following the internal user feedback session on 13/09/2023:

We presented the following prototype

(my-virtual-environment) ➜  kedro new 

Project Name
============
Please enter a human readable name for your new project.
Spaces, hyphens, and underscores are allowed. 
To skip this step in future use --name

[New Kedro Project]: My ML Pipeline 


Project Add-Ons 
================
Select which add-ons you'd like to include. 
To skip this step in future use --add-ons
To read more about these add-ons and what they do visit: kedro.org/

Add-Ons 
1) Lint:        Provides a basic linting set up with Black, Ruff 
2) Test:        Provides basic testing set up with pytest 
3) Log:         Provides more logging options, environment specific,  
4) Docs:        Provides basic documentations setup with Sphinx
6) PySpark:     Provides set up configuration for working with PySpark
8) Kedro-Viz:   Provides Kedro's native visualisation tool 

Which add-ons would you like to include in your project? [None/1-4/1,3/all]: 1,2

Example Pipeline 
================
Select whether you would like an example spaceflights pipeline included in your project.
To skip this step in the future use --example=y/n
To read more about how examples work visit: kedro.org/

Would you like to include an example pipeline? [Y\n]:Y

Congratulations! 
Your project My ML Pipeline has been created in directory /my_ml_pipeline 
You selected the following add-ons: Lint, Test 
It has been created with an example pipeline.

We received feedback on the following points: (❓flagged for further discussion)

1. the name add-ons makes it sound more optional, which can be misleading (possible names: tools, options, utilities, plugins) ❓
2. should we allow users to define preset groups of add-ons?
3. adjusting the copy of the CLI wizard to better highlight that we are recommending tools for certain functionality i.e. Black/Pytest instead of just Lint/Test ❓
4. an interactive playground/sandbox UI for generating kedro new commands in the docs. Allow people to experiment with options and preview what the project would look like.
5. users suggested that the default state should include examples, but no add-ons, as the most likely user type to use a basic new command is a new command. ❓
6. if the user does not select the testing add-on the example project should not contain tests. (especially because we wouldn't have installed the testing dependencies)
7. providing a custom flag to bypass the interactive flow and apply defaults to all flags that aren't provided. this will help for programmatic creation, and also expert users who want to skip the flow. ❓
8. the --starter flow should stay independent of the addons flow to allow users to start a project with their own custom project.

[amandakys]

Closed as initial design is now 'complete' and follow up work is tracked in #3054