Remove DeprecatedClassMeta in favor of getattr

[deepyaman]

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

Development notes

Checklist

• Read the contributing guidelines
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added a description of this change in the RELEASE.md file
• Added tests to cover my changes

[astrojuanlu]

Test failures on Windows are unrelated, I guess I'll restart the CI a couple more times...

[astrojuanlu]

Well, I can't reproduce these locally, but the win_e2e_tests are consistently failing every single time, always in the same scenario. Now I added a 120 seconds timeout (locally it takes ~1) and here's what happens:

  Scenario: Execute node convert into Python files                    # features/jupyter.feature:21
    Given I have prepared a config file                               # features/steps/cli_steps.py:143
    And I have run a non-interactive kedro new with starter "default" # features/steps/cli_steps.py:237
    Given I have added a test jupyter notebook                        # features/steps/cli_steps.py:226
    When I execute the test jupyter notebook and save changes         # features/steps/cli_steps.py:361
    And I execute the kedro jupyter command "convert --all"           # features/steps/cli_steps.py:330
    And Wait until the process is finished for up to "120" seconds    # features/steps/cli_steps.py:355
      Traceback (most recent call last):
        File "C:\tools\miniconda3\envs\kedro_builder\lib\site-packages\behave\model.py", line 1329, in run
          match.run(runner.context)
        File "C:\tools\miniconda3\envs\kedro_builder\lib\site-packages\behave\matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "features\steps\cli_steps.py", line 358, in wait
          context.result.wait(timeout)
        File "C:\tools\miniconda3\envs\kedro_builder\lib\subprocess.py", line 1189, in wait
          return self._wait(timeout=timeout)
        File "C:\tools\miniconda3\envs\kedro_builder\lib\subprocess.py", line 1473, in _wait
          raise TimeoutExpired(self.args, timeout)
      subprocess.TimeoutExpired: Command '['C:\\Users\\circleci\\AppData\\Local\\Temp\\tmpujtx_5z6\\Scripts\\kedro', 'jupyter', 'convert', '--all']' timed out after 120 seconds

    Then I should get a successful exit code                          # None
    And Code cell with node tag should be converted into kedro node   # None

I have absolutely no idea how the changes in this branch can interact with that part of the code, but it was working fine on main, and it failed always in the same way every time I restarted the CI here, so there must be something.

[deepyaman]

To ensure that dependent repos still pass:

• 🟢 kedro-viz
  • There's one mypy error that will probably need to be resolved after: package/tests/test_data_access/test_managers.py:233: error: Argument 2 to "isinstance" has incompatible type "AbstractDataSet[Any, Any]"; expected "Union[type, UnionType, Tuple[Union[type, UnionType, Tuple[Any, ...]], ...]]" [arg-type]
• 🟢 kedro-plugins
• 🟢 kedro-starters

[noklam]

The e2e test is failing due to timeout, not sure what is the issue yet. More importantly, I think this PR introduced some undesired behaviour.

See the screenshot below, doing any kedro command now will generate many warning message. This isn't the case with current main so I believe this is introduced by the PR.

[astrojuanlu]

That's true, we need to silence the deprecation warnings when importing the classes in our own code:

kedro/kedro/io/__init__.py

Line 6 in 345f6ac

from .cached_dataset import CachedDataSet, CachedDataset

[noklam]

Is it related? #2446

I have run the test locally in my window laptop and I don't have any problem.

[astrojuanlu]

Is it related? behave --no-capture -n "Execute node convert into Python files"

I can reproduce after SSH'ing to the CircleCI job, yes. However, running C:\Users\circleci\AppData\Local\Temp\...\Scripts\kedro jupyter convert --all works perfectly fine.

My suspicion is that, since there are more warnings now, the subprocess.Popen calls that our e2e tests use are getting deadlocked:

This will deadlock when using stdout=PIPE or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use Popen.communicate() when using pipes to avoid that.

https://docs.python.org/3/library/subprocess.html#subprocess.Popen.wait

I just made the change to .communicate to see if that fixes the issue.

[noklam]

Lookd promising, the convert tests is now passing, but the good old notebook and test started failing.
https://app.circleci.com/pipelines/github/kedro-org/kedro/23719/workflows/16441b18-1657-4d99-afc6-855e7065edbb/jobs/272755

[astrojuanlu]

Yep, I fixed one, but broke another. I consider it a success nonetheless 😂

In the short term, we should reduce the extra warnings to hopefully avoid the I/O deadlock. After that (in another PR) we might need to leave a note about reworking our e2e tests again. Supposedly subprocess.run takes care of killing the child processes.

But now... 💤

[astrojuanlu]

win_e2e tests are passing, will now go to sleep for real