Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Minimal support for LEGACY timeParserPolicy #2860

Closed
andygrove opened this issue Jul 1, 2021 · 0 comments · Fixed by #2875
Closed

[FEA] Minimal support for LEGACY timeParserPolicy #2860

andygrove opened this issue Jul 1, 2021 · 0 comments · Fixed by #2875
Assignees
Labels
feature request New feature or request

Comments

@andygrove
Copy link
Contributor

Is your feature request related to a problem? Please describe.
With spark.sql.legacy.timeParserPolicy set to LEGACY we currently fall back to CPU. This issue is for the initial work of supporting the legacy policy for format strings that are already supported in CORRECTED mode and which would work identically in LEGACY mode.

We will not support the hybrid Julian+Gregorian calendar that Spark supports in legacy mode so we need to document that and perhaps have a feature flag for enabling legacy support, since we are not fully compatible.

Describe the solution you'd like

  • With LEGACY mode enabled, and with supported formats, we should run on the GPU.
  • With LEGACY mode enabled, and with unsupported formats, we should fall back to CPU.

Describe alternatives you've considered
N/A

Additional context
N/A

@andygrove andygrove added feature request New feature or request ? - Needs Triage Need team to review and classify labels Jul 1, 2021
@andygrove andygrove added this to the June 21 - July 2 milestone Jul 1, 2021
@andygrove andygrove self-assigned this Jul 1, 2021
@andygrove andygrove removed the ? - Needs Triage Need team to review and classify label Jul 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant