[GraphBolt][CUDA] Full GPU support for GraphBolt

[mfbalin]

🔨Work Item

IMPORTANT:

• This template is only for dev team to track project progress. For feature request or bug report, please use the corresponding issue templates.
• DO NOT create a new work item if the purpose is to fix an existing issue or feature request. We will directly use the issue in the project tracker.

Project tracker: https://github.com/orgs/dmlc/projects/2

Required components

• Neighbor Sampling is being implemented without replacement and a homogenous case in [GraphBolt][CUDA] SampleNeighbors (Without replacement for now) #6770.
• Extend Neighbor Sampling to heterogeneous cases in [GraphBolt][CUDA] Add type_per_edge reading support and ensure same types are consecutive #6865.
• Extend Neighbor Sampling to heterogeneous cases with different fanouts for each edge type in [GraphBolt][CUDA] Multiple fanout hetero sampling support. #6895.
• [GraphBolt][CUDA] Ensure that edges with 0 probability are not sampled #6918
• Extend Neighbor Sampling for the sample with replacement case.
• Extend NeighborSampling for temporal sampling in [GraphBolt][CUDA][Temporal] Complete implementation. #7677.
• FusedCSCSamplingGraph::InSubgraph for CUDA in [GraphBolt][CUDA] InSubgraph #6830.
• IsIn operator in [GraphBolt][CUDA] gb.isin implementation #6829.
• [GraphBolt][CUDA] Enable GPU sampling in examples #6861.
• Enable all tests for the GPU backend.
• Add to() method to torch-based Feature and FeatureStore. #6901
• [GraphBolt][CUDA] Add native GPUCachedFeature instead of using DGL #6939
• [GraphBolt][CUDA] Showcase GPUCachedFeature usage in an example. #6955
• [GraphBolt][CUDA] Inplace pin memory for Graph and TorchFeatureStore #6962
• [GraphBolt][CUDA] MultiGPU example crashes #6978

Major optimizations

• [GraphBolt][CUDA] gb.expand_indptr #6871
• [GraphBolt][CUDA] Overlap feature fetcher #6954
• [GraphBolt] Refactor NeighborSampler and expose fine-grained datapipes. #6983 and [GraphBolt][CUDA] Pipelined sampling optimization #7039
• [GraphBolt][CUDA] Eliminate GPU synchronizations as much as we can. #6910
• [GraphBolt] Compress type_per_edge array so that instead of COO, we use CSC. #7143
• [GraphBolt][CUDA] Specialized kernels for simple Neighbor Sampling #7173
• [GraphBolt][CUDA] Optimized kernel for neighbor sampling general path #7272
• [GraphBolt][CUDA] Check if unique_and_compact can be optimized by using a hash table. #7174
• [GraphBolt][CUDA] Implement Cooperative Minibatching from arXiv:2310.12403 #7273

Miscellaneous

• [GraphBolt] Add more and larger BuiltinDatasets #6909
• [GraphBolt] Cannot re-initialize CUDA in forked subprocess #6981
• [GraphBolt] Refactor feature store #7274

@frozenbugs @peizhou001 @Rhett-Ying any operation I am missing above to enable full GraphBolt CUDA support?

[mfbalin]

@yxy235 Is there anything we are missing here when it comes to GPU support for GraphBolt?

[mfbalin]

@frozenbugs can we keep this issue as a tracker of the overall planned work for GraphBolt according to its design proposal? I plan to close it when the major proposed changes are done.

[mfbalin]

As you can see, there are unfinished tasks.