Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: excel sheet upload is not working #10450

Merged
merged 5 commits into from
Jul 29, 2020
Merged

Conversation

pphszx
Copy link
Contributor

@pphszx pphszx commented Jul 28, 2020

SUMMARY

The new excel sheet upload feature is not working, this issue is reported in #9825 (comment), this PR fixes it with come changes specific to pandas.read_excel.

e.g. args of chunksize, skipinitialspace are not supported, parse_dates is supported by both, filename arg is io, not filepath_or_buffer, which is for read_csv. sheet_name could be string, but integer should be more common.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Requires DB Migration.
  • Confirm DB Migration upgrade and downgrade tested.
  • Introduces new feature or API
  • Removes existing feature or API

@pphszx pphszx changed the title remove conflicts with csv upload fix excel sheet upload feature Jul 28, 2020
@pphszx pphszx changed the title fix excel sheet upload feature fix: excel sheet upload is not working Jul 28, 2020
@villebro
Copy link
Member

villebro commented Jul 28, 2020

Thanks @pphszx for the fix! The security manager methods were probably renamed while the original Excel upload PR was under review, hence this error sneaked in. See @10031 for details.

Are all the changes here necessary for making this work? If possible, I'd prefer to keep the changes to a minimum, and then follow up with a cleanup PR to add robustness. Not sure if you're aware, but we'll be releasing 0.37.0 soon, and it would be great to get this feature working for that release.

Ping @blcksrx : as you have most context, you're probably the best person to review this PR.

Copy link

@blcksrx blcksrx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @villebro mentioned above, I'd prefer to keep changes to be minimum.

superset/views/database/forms.py Outdated Show resolved Hide resolved
@pphszx
Copy link
Contributor Author

pphszx commented Jul 28, 2020

Hi @villebro, @blcksrx, I've add comments to all my changes, you can decide which part can be removed. You are free to change, just make this feature work on 0.37.0. Thanks!

@blcksrx
Copy link

blcksrx commented Jul 28, 2020

@pphszx Thanks for the description. I did read them all and still disagree with changing the sheet_name type from string to integer. what's your idea @villebro?

Copy link
Member

@bkyryliuk bkyryliuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General suggestion, let's add unit test to prevent regressions like this

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments. I'd also encourage adding a unit test as @bkyryliuk suggested, but understand it may be tricky due to the file format (one would probably need to programmatically generate an Excel file and then try to upload that with different variations of input values, asserting that the resulting table metadata is correct).

superset/views/database/forms.py Outdated Show resolved Hide resolved
superset/views/database/views.py Outdated Show resolved Hide resolved
Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for cleaning this up! @blcksrx are you ok with these changes?

@blcksrx
Copy link

blcksrx commented Jul 29, 2020

@villebro Yeah. I'm okay with that

@villebro villebro merged commit 473fe10 into apache:master Jul 29, 2020
villebro pushed a commit that referenced this pull request Jul 29, 2020
* remove conflicts with csv upload

* revert StringField

* change description

* remove redundant space

* apply string approach
@villebro villebro added the v0.37 label Jul 29, 2020
@pphszx pphszx deleted the fix_excel_upload branch July 29, 2020 07:50
auxten pushed a commit to auxten/incubator-superset that referenced this pull request Nov 20, 2020
* remove conflicts with csv upload

* revert StringField

* change description

* remove redundant space

* apply string approach
cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024
* remove conflicts with csv upload

* revert StringField

* change description

* remove redundant space

* apply string approach
@mistercrunch mistercrunch added 🍒 0.37.0 🍒 0.37.1 🍒 0.37.2 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.38.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/M v0.37 🍒 0.37.0 🍒 0.37.1 🍒 0.37.2 🚢 0.38.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants