Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Unify Tensors PR #6] Removed interfaces & members from lod_tensor,test=allcases #38811

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
e6c82df
Added shared_ptr<Allocation> member & corresponding interfaces to Sto…
jim19930609 Dec 21, 2021
23ce28d
Removed original pten::Allocation from Storage and adjusted the inter…
jim19930609 Dec 21, 2021
f668a9c
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 22, 2021
cbf5abb
Fixed issues with storage offset
jim19930609 Dec 22, 2021
d1562c8
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 23, 2021
5c67082
Used place to malloc allocation for TensorStorage
jim19930609 Dec 23, 2021
8c08dc6
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 23, 2021
c8f31ba
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 24, 2021
9151a1f
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 24, 2021
ff8783f
[Unify Tensors PR #3]Ported framework::Tensor interfaces to pten::Den…
jim19930609 Dec 26, 2021
3298585
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 27, 2021
235503f
Fixed issues with place
jim19930609 Dec 27, 2021
1ac5916
Added comments
jim19930609 Dec 28, 2021
705cf83
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 30, 2021
74dbfdb
Moved mutable_data with stream argument to DenseTensor
jim19930609 Dec 30, 2021
178447b
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 30, 2021
31a3014
Added set_offset interface
jim19930609 Dec 30, 2021
62f6c6a
Fixed CI issues,test=allcases
jim19930609 Dec 30, 2021
9cb9369
[Unify Tensors PR #4] Port LoDTensor interfaces to DenseTensor
jim19930609 Dec 28, 2021
4d5930e
Removed friend class EigenTensor/EigenMatrix/EigenVector from Tensor
jim19930609 Dec 30, 2021
515f128
Modified framework::Tensor to inherit from DenseTensor
jim19930609 Dec 30, 2021
23ddcdf
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 31, 2021
ab4e4a7
Reverted changes too pten_layout() interface
jim19930609 Dec 31, 2021
ab16975
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 31, 2021
093b9e8
Removed friend classes
jim19930609 Dec 31, 2021
b674632
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 31, 2021
f1ff906
Rearranged cfunction calls from tensor.data<void>() to tensor.data()
jim19930609 Dec 30, 2021
f03687a
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 31, 2021
c2f19bb
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Dec 31, 2021
4c14f73
Merge branch 'densetensor_interface_change_lod' of https://github.com…
jim19930609 Dec 31, 2021
c1098a1
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Jan 1, 2022
8056680
Merge branch 'densetensor_interface_change' of https://github.com/jim…
jim19930609 Jan 1, 2022
0e8dd04
Fixed CI issues
jim19930609 Jan 1, 2022
acd9f13
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Jan 4, 2022
08e1220
Fixed lite issues
jim19930609 Jan 4, 2022
36f6a39
Fixed data() interface issues,test=allcases
jim19930609 Jan 4, 2022
128f1f1
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Jan 5, 2022
de06d82
Resolved IsInitialized() issues
jim19930609 Jan 5, 2022
7c6b10f
Fixed ResetHolder() issues
jim19930609 Jan 5, 2022
1d38ca0
Fixed MKLDNN & Storage issues
jim19930609 Jan 5, 2022
3a4cfa9
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Jan 6, 2022
dadf29d
Resolved ShareBufferWith() issues
jim19930609 Jan 6, 2022
2d91053
Fixed LoD issues
jim19930609 Jan 7, 2022
f8c4a19
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Jan 7, 2022
9b62071
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Jan 7, 2022
239d57e
Merge branch 'tensor_inherit_from_dense_tensor' of https://github.com…
jim19930609 Jan 7, 2022
831866f
Removed interfaces & members from lod_tensor,test=allcases
jim19930609 Jan 8, 2022
5142751
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
jim19930609 Jan 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 1 addition & 51 deletions paddle/fluid/framework/lod_tensor.h
Original file line number Diff line number Diff line change
Expand Up @@ -108,64 +108,14 @@ bool CheckAbsLoD(const LoD& in, int tensor_height = -1);
*/
class LoDTensor : public Tensor {
public:
LoDTensor() : Tensor() {}

explicit LoDTensor(const LoD& lod) : lod_(lod) {}

void set_lod(const LoD& lod) { lod_ = lod; }

const LoD& lod() const { return lod_; }

LoD* mutable_lod() { return &lod_; }

/*
* Get the start offset and end offset of an element from LoD.
*/
std::pair<size_t, size_t> lod_element(size_t level, size_t elem) const {
PADDLE_ENFORCE_LT(
level, NumLevels(),
platform::errors::InvalidArgument(
"The input level of LoD is invalid, it should be less than LoD "
"size. The input level is %zu, the LoD size is %zu.",
level, NumLevels()));
PADDLE_ENFORCE_LT(elem, NumElements(level),
platform::errors::InvalidArgument(
"The input element of LoD is invalid, it should be "
"less than the number of elements in its level."
"The input element is %zu, the number of elements in "
"its level is %zu.",
elem, NumElements(level)));
return std::make_pair((lod_)[level][elem], (lod_)[level][elem + 1]);
}

/*
* Number of LoDTensor's levels, each level has units of data, for example,
* in the sentence's view, article, paragraph, sentence are 3 levels.
*/
size_t NumLevels() const { return lod_.size(); }
/*
* Number of elements in a level.
*/
size_t NumElements(size_t level = 0) const {
PADDLE_ENFORCE_LT(
level, NumLevels(),
platform::errors::InvalidArgument(
"The input level of LoD is invalid, it should be less than LoD "
"size. The input level is %zu, the LoD size is %zu.",
level, NumLevels()));
// the last offset is the end of last element
return (lod_)[level].size() - 1;
}
using Tensor::Tensor;

// Split LoDTensor and copy to each place specified in places.
std::vector<LoDTensor> SplitLoDTensor(
const std::vector<platform::Place> places) const;

void MergeLoDTensor(const std::vector<const LoDTensor*>& lod_tensors,
platform::Place place);

private:
LoD lod_;
};

/*
Expand Down
3 changes: 3 additions & 0 deletions paddle/fluid/framework/tensor.cc
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,10 @@ std::vector<Tensor> Tensor::Chunk(int64_t chunks, int64_t axis) const {

Tensor& Tensor::ShareDataWith(const Tensor& src) {
src.check_memory_size();
// Preserve LoD
auto lod = meta_.lod;
*this = src;
meta_.lod = lod;
return *this;
}
Tensor& Tensor::ShareInplaceVersionCounterWith(const Tensor& src) {
Expand Down