Decouple logging from ConfigLoader

[noklam]

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

Fix #2426

related:

• Update all starters to move conf/base/logging.yml to conf/logging.yml #2644 This has to be done later as we cannot change starters now.

Note. Breaking, goes in 0.19.0.

Following #2281 we have decided that users will no longer be able to change logging configuration dependent on kedro run environment. This is a huge simplification and means that logging only gets configured once in the kedro process with LOGGING = _ProjectLogging().

There's a few things to be done here, but they all need to be done at the same time or it will be very confusing.

• Remove KedroSession._setup_logging and KedroSession._get_logging_config 🥳
• Remove everything to do with logging from all config loaders (e.g. in config_patterns and docstring examples)
• Double check you've removed everything that needs removing by searching logging throughout the whole codebase. Note that configure_logging must stay since it's needed by the parallel runner
• Make sure you explain in the release notes migration guide that a custom logging file is now independent of kedro run environment and only used if KEDRO_LOGGING_CONFIG is set to point to it
• Related doc changes and example of logging.yml for advance users who may want to customise logging.

Development notes

Checklist

• Read the contributing guidelines
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added a description of this change in the RELEASE.md file
• Added tests to cover my changes

[astrojuanlu]

I see the target branch is develop ✔️ Does this fix all of gh-2426?

[noklam]

@astrojuanlu It's for 0.19.0 so it's in develop but not main. I planned to split the doc as a separate issue, but then I realised it's easier to have the doc in the same PR so I may just close #2645.

I haven't finished the doc change though.

[noklam]

Noted the develop branch is somehow outsync, I created this #2650 to manually fix it.

[antonymilne]

A few initial comments for now - I need to check through the codebase more carefully to see if there's anything else to do. But this looks great and is a huge improvement to kedro 💯

One thing I think is missing is that we can remove the logging.yml file from the project template now.

I think we should probably save the docs for a separate PR because it's a big rewrite, but up to you 🙂

[noklam]

@antonymilne I will leave the big doc changes for the separate PR and keep it minimal in this PR.

[noklam]

I have addressed the comments and remove the logging template

[antonymilne]

This is a huge improvement, thank you! 🙏 I would recommend picking up #2520 next so then we've got all the logging code changes complete and then write the docs after that.

When it comes to writing docs, we should be able to make the page MUCH simpler. Basically it would be something like:

1. By default kedro, uses rich for logging. For the full setup, look at the default_logging.yml file
2. If you want to change this then make your own logging.yml file. We recommend you put that file in conf. Then set KEDRO_CONFIG_LOGGING environment variable to point to it
3. Show some of the things you might like to change and the code needed to do it:
   a. Turn off rich tracebacks
   b. Turn on file logging
   c. Show DEBUG level messages

No need to explain the difference between framework-side and logging-side configuration or any of that confusing stuff any more 🎉

[noklam]

@antonymilne This sounds great! I think we may also need to revisit some deployment/databricks documentation too. I have open a ticket #2664

[ankatiyar]

Small typo here