Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: The function power_bi.execute_view should internally use the optimized version for getting data, but still retain the same shape #888

Closed
Cubewise-gtejeda opened this issue Apr 5, 2023 · 1 comment · Fixed by #893

Comments

@Cubewise-gtejeda
Copy link

When comparing cells.execute_mdx_csv(use_iterative_json=True) to power_bi.execute_view on a larger dataset, it shows an around 85% memory improvement when using execute_mdx_csv.
The two dimensional cube used for this test had one measure on the column and 5 million rows. There was no data in the cube and no zero suppression had been applied.

power_bi.execute_view:

image

execute_mdx_csv(use_iterative_json=True):

image

@MariusWirtz
Copy link
Collaborator

#893 solves this.
When working with very large data sets, you can now use execute_view_dataframe(..., shaped=True) instead of execute_view_dataframe_shaped.

You can also pass use_iterative_json=True or use_blob=True to the existingtm1.cells.execute_view_dataframe_shaped and tm1.power_bi.execute_view function.
Please note that in this case, the resulting data frame will be similar but slightly different:

  • types are inherited slightly differently
  • empty rows are omitted by default (unless skip_zeros=False is passed)
  • 0 values in cells instead of nan

Sample:

from TM1py import TM1Service

params = {
    "address": "",
    "port": 8010,
    "user": "admin",
    "password": "apple",
    "ssl": True
}

with TM1Service(**params) as tm1:
    df = tm1.power_bi.execute_view("Sales Cube", "Default", private=False)
    print(df.to_markdown())

    df = tm1.power_bi.execute_view("Sales Cube", "Default", private=False, use_iterative_json=True)
    print(df.to_markdown())

    df = tm1.power_bi.execute_view("Sales Cube", "Default", private=False, use_blob=True)
    print(df.to_markdown())

    df = tm1.power_bi.execute_view("Sales Cube", "Default", private=False, use_blob=True, skip_zeros=False)
    print(df.to_markdown())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants