Add support for multiple data types (will break up into smaller pull requests) #196

hcho3 · 2020-08-29T07:54:13Z

Addresses #95 and #111.

Trying again, since #130 failed. This time, I made the Model class to be polymorphic. This way, the amount of pointer indirection is minimized.

Summary: Model is an opaque container that wraps the polymorphic handle ModelImpl<ThresholdType, LeafOutputType>. The handle in turn stores the list of trees Tree<ThresholdType, LeafOutputType>. To unbox the Model container and obtain ModelImpl<ThresholdType, LeafOutputType>, use Model::Dispatch(<lambda expression>).

Also, upgrade to C++14 to access the generic lambda feature, which proved to be very useful in the dispatching logic for the polymorphic Model class.

EDIT. I will break up this PR into smaller PRs, once I get the whole system working together correctly.

TODOs

Turn the Model and Tree classes into template classes
Revise the string templates so that correct data types are used in the generated C code
Rewrite the model builder class
Revise the zero-copy serializer
Create an abstract matrix class that supports multiple data types (float32, float64 for now).
Redesign the C runtime API, using the abstract matrix class.
Ensure accuracy of scikit-learn models. To achieve the best results, use float32 for the input matrix and float64 for the split thresholds and leaf outputs.
Revise the JVM runtime.

…ss with all possible template args

…mlc#196) (dmlc#199) * New prediction runtime C API, to support multiple data types * Address reviewer's feedback

) (dmlc#201) * Upgrade C++ standard to C++14. * Split struct Model into class Model and class ModelImpl. The ModelImpl class will soon become a template class in order to hold Tree objects with uint32, float32, or float64 type. The Model class will become an abstract class so as to avoid exposing ModelImpl to external interface. (It's very hard to pass template classes through a FFI boundary.) * Change signature of methods that return Model, since Model is now an abstract class. These functions now return std::unique_ptr<Model>. * Move bodies of tiny methods from tree_impl.h to tree.h. This will reduce verbosity once ModelImpl becomes a template class.

…o multi_type_support2

@canonizer

… data types (Part of dmlc#196) (dmlc#198) * Create template classes for model representations to support multiple data types * Update include/treelite/tree.h Co-authored-by: William Hicks <wphicks@users.noreply.github.com> * Address review comments from @canonizer * Address more comments from @canonizer Co-authored-by: William Hicks <wphicks@users.noreply.github.com> Co-authored-by: William Hicks <wphicks@users.noreply.github.com> Co-authored-by: Andy Adinets <aadinets@nvidia.com>

…o multi_type_support2

hcho3 · 2020-09-25T09:37:33Z

#198, #199, #201, and #203 have been merged into a separate branch multi_type_support_refactor. Once the XGBoost JSON parser (#202) is merged into the mainline, the refactor branch will be also merged into the mainline.

…196) (#199) * The runtime now queries the data type for the model it loads, via QueryThresholdType() and QueryLeafOutputType(). Every compiled model now embeds the type information. * Implement a full-fledged data matrix class DMatrix in the runtime, to replace DenseBatch and SparseBatch. The former *Batch classes assumed float32 data, whereas the new DMatrix class is able to handle both float32 and float64. * Some API functions like PredictBatch() now takes void* pointers to accommodate multiple data types. Co-authored-by: Yuta Hinokuma <higumachan@users.noreply.github.com>

…#201) * Upgrade C++ standard to C++14. * Split struct Model into class Model and class ModelImpl. The ModelImpl class will soon become a template class in order to hold Tree objects with uint32, float32, or float64 type. The Model class will become an abstract class so as to avoid exposing ModelImpl to external interface. (It's very hard to pass template classes through a FFI boundary.) * Change signature of methods that return Model, since Model is now an abstract class. These functions now return std::unique_ptr<Model>. * Move bodies of tiny methods from tree_impl.h to tree.h. This will reduce verbosity once ModelImpl becomes a template class. Co-authored-by: Andy Adinets <aadinets@nvidia.com>

… data types (Part of #196) (#198) * The ModelImpl class (created in #201) becomes the template class ModelImpl<ThresholdType, LeafOutputType>. * Implement template classes ModelImpl<ThresholdType, LeafOutputType> and TreeImpl<ThresholdType, LeafOutputType> that contain the details of the tree ensemble model. The template classes are parameterized by the types of the thresholds and leaf outputs. Currently the following combinations are allowed: | Threshold type | Leaf output type | |----------------|------------------| | float32 | float32 | | float32 | uint32 | | float64 | float64 | | float64 | uint32 | |----------------|------------------| * Revise the zero-copy serialization protocol, to prepend the type information (threshold_type, leaf_output_type) to the serialized types so that the recipient will choose the correct ModelImpl<ThresholdType, LeafOutputType> to deserialize to. * A run-time type dispatching system using the enum type TypeInfo. Users are able to dispatch a correct version of ModelImpl<ThresholdType, LeafOutputType> by specifying a pair of TypeInfo values. We also implement a set of convenient functions, such as InferTypeInfoOf<T> that converts the template arg T into TypeInfo enum. Co-authored-by: William Hicks <wphicks@users.noreply.github.com> Co-authored-by: Andy Adinets <aadinets@nvidia.com>

Addresses #95 and #111. Follow-up to #198, #199, #201 Trying again, since #130 failed. This time, I made the Model class to be polymorphic. This way, the amount of pointer indirection is minimized. Summary: Model is an opaque container that wraps the polymorphic handle ModelImpl<ThresholdType, LeafOutputType>. The handle in turn stores the list of trees Tree<ThresholdType, LeafOutputType>. To unbox the Model container and obtain ModelImpl<ThresholdType, LeafOutputType>, use Model::Dispatch(<lambda expression>). Also, upgrade to C++14 to access the generic lambda feature, which proved to be very useful in the dispatching logic for the polymorphic Model class. * Turn the Model and Tree classes into template classes * Revise the string templates so that correct data types are used in the generated C code * Rewrite the model builder class * Revise the zero-copy serializer * Create an abstract matrix class that supports multiple data types (float32, float64 for now). * Move the DMatrix class to the runtime. * Extend the DMatrix class so that it can hold float32 and float64. * Redesign the C runtime API using the DMatrix class. * Ensure accuracy of scikit-learn models. To achieve the best results, use float32 for the input matrix and float64 for the split thresholds and leaf outputs. * Revise the JVM runtime.

hcho3 marked this pull request as draft August 29, 2020 07:54

hcho3 force-pushed the multi_type_support2 branch 3 times, most recently from 4458ec7 to 322acc8 Compare August 29, 2020 08:06

Make Model and Tree classes into template classes

2e823eb

hcho3 force-pushed the multi_type_support2 branch from 322acc8 to 2e823eb Compare August 29, 2020 22:24

hcho3 added 2 commits August 30, 2020 01:18

Add back size check for Node class; define format string for Node cla…

edcd1f4

…ss with all possible template args

Update ModelBuilder interface + zero-copy serializer

c0d1abd

hcho3 force-pushed the multi_type_support2 branch from ad3314a to c0d1abd Compare August 31, 2020 09:52

Update C interface for model builder API

acc20be

hcho3 force-pushed the multi_type_support2 branch from 5ec551a to acc20be Compare August 31, 2020 18:58

hcho3 added 3 commits August 31, 2020 12:41

Fix lint checks

34c2231

Add a missing header

86104d6

Perform round-trip serialization test with all possible types

77477e7

hcho3 force-pushed the multi_type_support2 branch from 3f3ec0d to 77477e7 Compare August 31, 2020 21:51

Update native code template

b846f6a

hcho3 force-pushed the multi_type_support2 branch from ab6277d to b846f6a Compare August 31, 2020 23:30

hcho3 added 4 commits August 31, 2020 20:22

Move TypeInfo to a separate header + objtreelite_common

0f88e96

Move data.cc to objtreelite_common; start new DMatrix design

81e9c7c

New DMatrix design

e4bb3cb

Use new DMatrix in branch annotator; move DMatrix to objtreelite_common

f64ddc8

hcho3 force-pushed the multi_type_support2 branch from a658f01 to 4515d9d Compare September 2, 2020 04:43

hcho3 added 4 commits September 1, 2020 23:15

Make ModelImpl a derived class of Model, simplifying dispatch

1c74cc5

Add functions to query data types used in a compiled predictor

e8f874f

Remove single-instance prediction feature from the runtime

71d43aa

Refactor the runtime API to use DMatrix

914469b

hcho3 force-pushed the multi_type_support2 branch from 4515d9d to 914469b Compare September 3, 2020 02:37

hcho3 added 3 commits September 3, 2020 07:42

Get moder builder and runtime API working end-to-end

13ee2b6

Emit correct data type for leaf outputs

7aef6db

Remove Model Type; consolidate type dispatching logic

f2715d9

This was referenced Sep 10, 2020

Create template classes for model representations to support multiple data types (Part of #196) #198

Merged

New prediction runtime C API, to support multiple data types (Part of #196) #199

Merged

hcho3 added 2 commits September 10, 2020 00:33

Remove a void* handle that's unused

b1e1e4b

Merge remote-tracking branch 'origin/mainline' into multi_type_support2

e149dc4

hcho3 marked this pull request as ready for review September 10, 2020 07:40

hcho3 closed this Sep 10, 2020

hcho3 reopened this Sep 10, 2020

hcho3 mentioned this pull request Sep 14, 2020

Refactor struct Model -> class Model + class ModelImpl (Part of #196) #201

Merged

hcho3 added 4 commits September 15, 2020 00:25

New prediction runtime C API, to support multiple data types (Part of d…

6421fa2

…mlc#196) (dmlc#199) * New prediction runtime C API, to support multiple data types * Address reviewer's feedback

Merge remote-tracking branch 'origin/multi_type_refactor_breakup' int…

fdcb519

…o multi_type_support2

Inline GetThresholdType() and GetLeafOutputType()

8138e13

hcho3 mentioned this pull request Sep 24, 2020

[FEA] Raise a warning if LightGBM 64-bit thresholds may cause rounding issues rapidsai/cuml#2817

Closed

hcho3 and others added 2 commits September 25, 2020 01:04

Merge remote-tracking branch 'origin/multi_type_refactor_breakup' int…

d9173df

…o multi_type_support2

hcho3 force-pushed the multi_type_support2 branch from 65d83ee to d9173df Compare September 25, 2020 08:47

hcho3 closed this Sep 25, 2020

hcho3 deleted the multi_type_support2 branch September 25, 2020 09:37

hcho3 mentioned this pull request Oct 9, 2020

Option for 64bit input data #95

Closed

hcho3 mentioned this pull request Oct 24, 2020

Different score after compiling a lightgbm model #115

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for multiple data types (will break up into smaller pull requests) #196

Add support for multiple data types (will break up into smaller pull requests) #196

hcho3 commented Aug 29, 2020 •

edited

Loading

hcho3 commented Sep 25, 2020

Add support for multiple data types (will break up into smaller pull requests) #196

Add support for multiple data types (will break up into smaller pull requests) #196

Conversation

hcho3 commented Aug 29, 2020 • edited Loading

hcho3 commented Sep 25, 2020

hcho3 commented Aug 29, 2020 •

edited

Loading