Adjusting HBase Support

[graben1437]

Remove back level versions of HBase that are not designed for Hadoop 2.
If someone really needs these very old versions and are re-building them for Hadoop 2,
then they can also rebuild Titan for these old versions.

[dalaro]

#1225 introduced a one-line conflict with this, but it would be trivial to resolve. You don't need to rebase just for that one change.

I was thinking about this over the weekend. Dropping 0.96 would be fine with me. No love lost there. Whether 0.94 is really and truly dead yet is not as obvious to me, considering the special niche it fills (supporting ancient deployments that people can't take down to migrate).

This happened a year ago, but it was inconclusive: http://mail-archives.apache.org/mod_mbox/hbase-user/201412.mbox/%3C1351645080.407883.1418669587849.JavaMail.yahoo@jws10685.mail.bf1.yahoo.com%3E

I read this when it was posted, but it's both sort of old (from February) and doesn't seem to say exactly what's happening to 0.94. The chart is suggestive, but I think the text is silent. https://blogs.apache.org/hbase/entry/start_of_a_new_era

And the most recent HBase 0.94 release postdates both of those things (0.94.27 on Mar 25).

I really do want to drop HBase 0.94 support when it dies upstream, since it will immediately cull a bunch of old dependencies, including our single oldest Guava version dependency (IIRC). I'm just not sure it's actually dead yet. It seems to be undead, but maybe I just missed the memo. Any thoughts or links about that?

[graben1437]

Since TinkerPop 3 exclusively supports Hadoop 2, I was trying to reduce the Titan support matrix to more current versions designed for Hadoop 2 with this pull request to keep things sane.

While it is possible to build HBase 0.94 to work with Hadoop 2, per this links suggestion (I have not tried it):
[
http://hbase.apache.org/0.94/book.html
..are you trying to talk to an Hadoop 2.0.x from an HBase that has an Hadoop 1.0.x client? Use the HBase built against Hadoop 2.0 or rebuild your HBase passing the -Dhadoop.profile=2.0
]

The following thoughts guided this proposal:
a) it seems fair to assume if someone is still on HBase 0.94 that they are using an older version of Titan too.
b) It isn't clear HBase 0.94 can participate in the OLAP pieces of Titan (I didn't test this but suspect this to be the case), which means Titan+TP3.x isn't really fully supporting HBase 0.94 anyway.

The vast dependency matrix for Titan makes the build complex and the testing and support even more complex and expensive. For example, what are we to now do about Solr 5.4 which has breaking API changes ? What is your guideline on adding/taking away support for older versions of dependencies ? I suggest we might need a more cut and dry policy (beyond what we think is being used) to make things saner. Does that make any sense or is that just babble ?

[graben1437]

I have been running with these proposed changes in a forked build that is equivalent to Titan 1.1.x with the latest TinkerPop support (3.1.0, 3.1.1) successfully now for several weeks. Since no changes are being pulled into Titan, it isn't clear anymore what the state of the main branch of Titan is with regard to these changes. But these changes make nice matrix clean-ups and add HBase improvements that work for me with OLTP and OLAP.

[sjudeng]

Hi. As suggested in #1269, I tried to see if #1268 was resolved in this branch but I had issues trying to run tests, specifically in titan-hadoop-2. Also note there were two dependency definitions (hbase-094 and hbase-096) that needed to be removed from titan-hbase to get the branch compiling.

See below for the test error. It looks like it might be a dependency issue relating to jersey-server. Any thoughts?

git remote add graben1437 https://github.com/graben1437/titan.git
git fetch graben1437
git checkout hbase-support1
mvn clean install -DskipTests -Phadoop2 -Dmaven.repo.local=$HOME/.m2/graben1437
mvn verify -Phadoop2 -Dmaven.repo.local=$HOME/.m2/graben1437 -Dit.test="**/CassandraInputFormatIT.java" -rf :titan-hadoop-2
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 42.543 sec <<< FAILURE! - in com.thinkaurelius.titan.hadoop.CassandraInputFormatIT
testReadGraphOfTheGods(com.thinkaurelius.titan.hadoop.CassandraInputFormatIT)  Time elapsed: 33.964 sec  <<< ERROR!
java.lang.IllegalStateException: java.lang.NoClassDefFoundError: com/sun/jersey/spi/container/servlet/ServletContainer
    at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
    at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
    at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ComputerResultStep.processNextStart(ComputerResultStep.java:80)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:126)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:37)
    at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.next(DefaultTraversal.java:154)
    at com.thinkaurelius.titan.hadoop.CassandraInputFormatIT.testReadGraphOfTheGods(CassandraInputFormatIT.java:30)
Caused by: java.lang.NoClassDefFoundError: com/sun/jersey/spi/container/servlet/ServletContainer

[sjudeng]

The test error was a problem in the source branch, tp3-ci-31 and it looks like it's fixed in titan11. I merged and am testing issue #1268.