Skip to content
This repository has been archived by the owner on Dec 20, 2018. It is now read-only.

Add support for Spark 2.1.x (while retaining Spark 2.0.x support) #212

Closed
wants to merge 1 commit into from

Conversation

JoshRosen
Copy link
Contributor

@JoshRosen JoshRosen commented Feb 7, 2017

This patch builds on #206 in order to restore support for Spark 2.0.x. This means that a single binary artifact can be used with both Spark 2.0.x and 2.1.x, simplifying the builds of downstream projects which are compatible with both Spark versions.

This was referenced Feb 7, 2017
@JoshRosen JoshRosen added this to the 3.2.0 milestone Feb 7, 2017
@codecov-io
Copy link

codecov-io commented Feb 7, 2017

Codecov Report

Merging #212 into branch-3.2 will increase coverage by 0.78%.

@@              Coverage Diff               @@
##           branch-3.2     #212      +/-   ##
==============================================
+ Coverage       90.62%   91.41%   +0.78%     
==============================================
  Files               5        5              
  Lines             320      326       +6     
  Branches           49       54       +5     
==============================================
+ Hits              290      298       +8     
+ Misses             30       28       -2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b988184...78f3a34. Read the comment docs.

@bdrillard
Copy link

A pull request for #169 would require the latest changes in the ObjectType API, which are only reflected in Spark 2.1.x. I think that would mean that a version of Spark-Avro with code for #169 merged, but without explicitly Spark 2.1.x, would break if the user tried to use the feature for datasets.

@JoshRosen
Copy link
Contributor Author

@bdrillard, my plan was to release a cross-compatible version as 3.2.0 now and then to revisit dropping 2.0.x compatibility and bumping the version number to 4.0.0 in master. In fact, let me go ahead and merge @koertkuipers's PR into master and then fork a branch-3.2 which adds my compatibility changes.

@JoshRosen JoshRosen changed the base branch from master to branch-3.2 February 7, 2017 23:33
@bdrillard
Copy link

@JoshRosen that makes sense to me. I'll prepare a PR for #169, and I suppose we can make sure it gets merged into a branch/release that ensures the correct version of Spark.

JoshRosen added a commit that referenced this pull request Feb 7, 2017
This patch builds on #206 in order to restore support for Spark 2.0.x. This means that a single binary artifact can be used with both Spark 2.0.x and 2.1.x, simplifying the builds of downstream projects which are compatible with both Spark versions.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #212 from JoshRosen/add-spark-2.1.
@JoshRosen JoshRosen closed this Feb 7, 2017
@JoshRosen JoshRosen deleted the add-spark-2.1 branch February 7, 2017 23:41
@JoshRosen
Copy link
Contributor Author

I'll publish a new release as soon as I'm in an environment with access to my GPG signing key.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants