[jvm-package] SEGV in spark predict with KryoSerializer

[raskr]

@hcho3 (cc: @sperlingxx)

Hi, I tested out spark runtime using TreeLiteMode in yarn cluster, and observed JNI call segfaulting.

Let me get straight to the point, for JavaSerializer which is a slow one not commonly used in spark, Serializable implementation in
Preditor.java is sufficient but is not for KryoSerializer. It should implement KryoSerializable as well:

public class Predictor implements Serializable, KryoSerializable {
...

I'll submit a patch fixing this later if there is not problem.

[hcho3]

Thanks!

[hcho3]

Is the issue related to #163 ?

[raskr]

Im' not sure. In my case, Predictor object containing a handle to a native runtime was not reconstructed in spark runtime as expected, then segfaulted. The scenario was:

1. spark driver node initializes the native binary and runtime
2. the driver serializes and distributes the binary to worker nodes
3. worker nodes try to deserialize the Predictor instance using KryoSerializer
4. Kryo doesn't call readObject and uses the default constructor instead to reconstruct it, then this.handle remains 0
5. any call using TreeliteJNI with handle=0 results in SEGV. (handle=0 is treated as null pointer in C++)