-
Notifications
You must be signed in to change notification settings - Fork 167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some cleanups, fix memoryview support #122
Conversation
Codecov Report
@@ Coverage Diff @@
## master #122 +/- ##
==========================================
- Coverage 86.32% 83.94% -2.38%
==========================================
Files 2 2
Lines 541 517 -24
Branches 98 90 -8
==========================================
- Hits 467 434 -33
- Misses 55 62 +7
- Partials 19 21 +2
Continue to review full report at Codecov.
|
Predictably, Python 2.6 fails on memoryview. Should I fix this PR or should we ditch 2.6 (and 3.3)? :-) |
How about we finish making the release for |
Works for me :-) |
I'm going to give @ssanderson a day or so to make the release then I'll ping again and go from there. 😉 |
+1 to dropping 2.6 and 3.3. Here's the pip installs for cloudpickle from PyPI for the last month -- they have very small numbers!
|
We have more Python3 downloads than Python2 downloads?!?! 😮 |
Woot
…On Thu, Oct 26, 2017 at 5:32 PM, Kyle Kelley ***@***.***> wrote:
We have more Python3 downloads than Python2 downloads?!?! 😮
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#122 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AASszOVhifl_SkGpm9YqHWOJpYPuvAlZks5swPpcgaJpZM4QIAH7>
.
|
Alright, adjust that Travis matrix to get rid of 2.6 support and we can bring this in. |
## What changes were proposed in this pull request? The version of cloudpickle in PySpark was close to version 0.4.0 with some additional backported fixes and some minor additions for Spark related things. This update removes Spark related changes and matches cloudpickle [v0.4.3](https://github.com/cloudpipe/cloudpickle/releases/tag/v0.4.3): Changes by updating to 0.4.3 include: * Fix pickling of named tuples cloudpipe/cloudpickle#113 * Built in type constructors for PyPy compatibility [here](cloudpipe/cloudpickle@d84980c) * Fix memoryview support cloudpipe/cloudpickle#122 * Improved compatibility with other cloudpickle versions cloudpipe/cloudpickle#128 * Several cleanups cloudpipe/cloudpickle#121 and [here](cloudpipe/cloudpickle@c91aaf1) * [MRG] Regression on pickling classes from the __main__ module cloudpipe/cloudpickle#149 * BUG: Handle instance methods of builtin types cloudpipe/cloudpickle#154 * Fix <span>#</span>129 : do not silence RuntimeError in dump() cloudpipe/cloudpickle#153 ## How was this patch tested? Existing pyspark.tests using python 2.7.14, 3.5.2, 3.6.3 Author: Bryan Cutler <cutlerb@gmail.com> Closes apache#20373 from BryanCutler/pyspark-update-cloudpickle-42-SPARK-23159.
A couple assorted cleanups:
save_global
can use the default implementation most of the timesave_buffer
andsave_memory
needn't callPickler.save_string
directlysave_reduce
instead of dumping opcodes by hand where possibleAlso, memoryviews should obviously serialize as bytes, not str.