Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add read_avro and list_avro_columns for rework on Splittable Avro sup…
…port (#399) This PR is part of the effort to rework on Dataset with large files reading into Tensors first to speed up performance. See 382 and 366 for related discussions. Summary: 1) read_avro is able to read a avro file within the range of [offset, offset+length] (Splittable) 2) we use primitive read_avro C++ ops to read in big chunks and then wire up with tf.data.Dataset 3) read_avro could be used in other places. 4) AvroDataset automatically find out the dtype in eager mode, in graph mode, user has to specify the dtype in kwargs. Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
- Loading branch information