FIX-#4023: Fall back to pandas in case of MultiIndex columns

[AndreyPavlenko]

MultiIndex columns are not currently supported by HDK.

What do these changes do?

• first commit message and PR title follow format outlined here

  NOTE: If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

• passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
• passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
• signed commit with git commit -s
• Resolves dataframe.rename doesn`t work correctly at MODIN_STORAGE_FORMAT=omnisci #4023?
• tests added and passing
• module layout described at docs/development/architecture.rst is up-to-date

[dchigarev]

why this was removed? why can't we test multi-index columns for .reset_index and .renme at the same time?

[AndreyPavlenko]

Because, in case of MultiIndex columns, it falls back to Pandas. The test fails, in this mode, due to the forced lazy execution. MultiIndex renaming is not supported for columns, but it's supported for rows. I think, it makes more sense to test this functionality, rather than default pandas functionality.

[dchigarev]

So previously this test used to pass successfully despite the columns being MultiIndex? I wonder then whether it's a proper solution to mark the whole frame as has_unsupported_data=True if it's only a thing for the .rename() and maybe some other subset of functions? Shouldn't we only mark .rename() and those other methods that can't handle MultiIndex columns as falling back to pandas and leave the others untouched since they can process this with no problems?

[AndreyPavlenko]

The test passed just because it does nothing with columns. In fact, a frame with MultiIndex columns is totally broken due to column names conversion to HDK-supported format. It's not even possible to print such a frame. If we mark only subset of operations, as you suggested, there could be a situation, when some supported operations applied and when an unsupported operation is called, it will not be able to fall back to pandas, because to_pandas() fails for such frames.

After the fix, this test failed due to forced lazy execution. I could set force_lazy=False in the test, but, I think, it does not make any sense. If we have MultiIndex columns, we are testing pandas, but not modin. That's why I've removed the multiindex columns.