Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement the type of data in channel #8284

Closed
chengduoZH opened this issue Feb 8, 2018 · 3 comments
Closed

Implement the type of data in channel #8284

chengduoZH opened this issue Feb 8, 2018 · 3 comments
Assignees

Comments

@chengduoZH
Copy link
Contributor

Currently, the function of Channel of Fluid has been nearly completed. But to use Channel in Fluid, we also need define the data type of Channel.
I think the data type of it should be n-tuple, just like std::tuple. For example, we want to send (image, label) using one channel, rather than using multiple channels.
The following is what I complete ChanDesc, it includes the base-data-type and advanced-data-type.

message MetaData {
  enum DataType {
    BOOL = 1;
    INT16 = 2;
    INT32 = 3;
    INT64 = 4;
    FP16 = 5;
    FP32 = 6;
    FP64 = 7;
    TENSOR = 8;
    LOD_TENSOR = 9;
    SELECTED_ROWS = 10;
    LOD_TENSOR_ARRAY = 11;
    // or others 
  }
  required DataType base_type = 1;
  optional bool data_bool = 2;
  optional int32 data_int32 = 3;
  optional int64 data_int64 = 4;
  optional float data_float = 5;
  optional LoDTensorDesc lod_tensor = 6;
  optional TensorDesc selected_rows = 7;
  optional LoDTensorArrayDesc tensor_array = 8;
}

message ChanEleDesc { repeated MetaData meta_data = 1; }

message ChanDesc {
  repeated ChanEleDesc channel_type = 1;
  required int32 cap = 2 [ default = 0 ];
  required string name = 3;
}
@chengduoZH
Copy link
Contributor Author

We can replace MetaData with VarDesc, and VarDesc includes ChanDesc.

message ChanEleDesc { repeated VarDesc meta_data = 1; }

message ChanDesc {
  repeated ChanEleDesc channel_type = 1;
  required int32 cap = 2 [ default = 0 ];
  required string name = 3;
}

message VarDesc {
  enum VarType {
    LOD_TENSOR = 1;
    SELECTED_ROWS = 2;
    FEED_MINIBATCH = 3;
    FETCH_LIST = 4;
    STEP_SCOPES = 5;
    LOD_RANK_TABLE = 6;
    LOD_TENSOR_ARRAY = 7;
    PLACE_LIST = 8;
    READER = 9;
    CHAN = 10;
  }
  required string name = 1;
  required VarType type = 2;
  optional bool persistable = 3 [ default = false ];
  optional LoDTensorDesc lod_tensor = 4;
  optional TensorDesc selected_rows = 5;
  optional LoDTensorArrayDesc tensor_array = 6;
  optional ReaderDesc reader = 7;
  optional ChanDesc chan = 8;
}

@helinwang
Copy link
Contributor

Thanks for initiating the discussion!

Maybe rename channel_type to element_type, since it's about the type of each element in the tuple, not the type of channel.

And, what are


  optional bool data_bool = 2;
  optional int32 data_int32 = 3;
  optional int64 data_int64 = 4;
  optional float data_float = 5;

used for? I thought the MetaData is only for describing the channel elements, not for storing the data.

@chengduoZH
Copy link
Contributor Author

Thanks for your comment!

I thought the MetaData is only for describing the channel elements, not for storing the data.

You are right, this issue is too old. To support the type of Channel, Fluid's type system should be redefined, this PR(#8352) is working for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants