Add copy from tensor

[shangzhizhou]

PR types

Others

PR changes

APIs

Describe

add copy_from_tensor api for inference tensor

一、python增加同步copy_tensor的api，使用示例

from paddle.inference.contrib import utils

utils.copy_tensor(dst_tensor, src_tensor)

二、C++增加paddle_infer::Tensor::CopyToCpuAsync接口(当前只支持GPU到CPU的异步拷贝，此时不能使用Host申请的内存，需要使用cuda的pinned memory，可以使用提供的工具函数申请和释放 CudaMallocPinnedMemory()/CudaFreePinnedMemory())，示例如下

1. 返回stream的调用方式

  //...
  //predictor运行代码

  const auto &output_names = predictor->GetOutputNames();
  auto output_tensor = predictor->GetOutputHandle(output_names[0]);
  std::vector<int> output_shape = output_tensor->shape();
  int out_num = std::accumulate(output_shape.begin(), output_shape.end(), 1,
                                std::multiplies<int>());

  float *out_data = static_cast<float *>(
      contrib::TensorUtils::CudaMallocPinnedMemory(sizeof(float) * out_num));

  cudaStream_t stream;
  output_tensor->CopyToCpuAsync(out_data, static_cast<void *>(&stream));

  // sync
  cudaStreamSynchronize(stream);

  contrib::TensorUtils::CudaFreePinnedMemory(static_cast<void *>(out_data));

2. 使用回调的调用方式

  //...
  //predictor运行代码

  const auto &output_names = predictor->GetOutputNames();
  auto output_tensor = predictor->GetOutputHandle(output_names[0]);
  std::vector<int> output_shape = output_tensor->shape();
  int out_num = std::accumulate(output_shape.begin(), output_shape.end(), 1,
                                std::multiplies<int>());

  float *out_data = static_cast<float *>(
      contrib::TensorUtils::CudaMallocPinnedMemory(sizeof(float) * out_num));

  output_tensor->CopyToCpuAsync(
      out_data,
      [](void *cb_params) {
        float *data = static_cast<float *>(cb_params);
        for (int i = 0; i < 10; i++) {
          std::cout << data[i] << std::endl;
        }
      },
      static_cast<void *>(out_data));

  cudaDeviceSynchronize();
  contrib::TensorUtils::CudaFreePinnedMemory(static_cast<void *>(out_data));

三、增加C++ tensor拷贝函数

  static void CopyTensor(Tensor* p_dst, const Tensor& src);
  static void CopyTensorAsync(Tensor* p_dst, const Tensor& src,
                              void* exec_stream);
  static void CopyTensorAsync(Tensor* p_dst, const Tensor& src, CallbackFunc cb,
                              void* cb_params);

异步使用方式参考Tensor.CopyToCpuAsync()
测试代码参考 paddle/fluid/inference/tests/api/paddle_infer_api_copy_tensor_tester.cc

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[Superjomn]

Imp -> Impl

[shangzhizhou]

done

[Superjomn]

input_tensor

_t 后缀一般表示 type，比如 value_t

[shangzhizhou]

done

[Superjomn]

全局， Imp -> Impl

[shangzhizhou]

done

[Superjomn]

用于单测的，最好单独拆出去，用 WITH_TESTING 宏隔开

这个会进到最终生产环境的库里面吧？

[shangzhizhou]

done，thanks

[Superjomn]

文件名 copy_tensor，两个单词

[Superjomn]

这是个单测？

文件名也需要加 _tester

[shangzhizhou]

done

[Superjomn]

最好把用例，功能在 PR 描述里面也加下

[shangzhizhou]

最好把用例，功能在 PR 描述里面也加下

done

[XieYunshen]

LGTM for 'set_tests_properties(paddle_infer_api_copy_tensor_tester PROPERTIES TIMEOUT 30) '