Work on Embedded

[hedaoyuan]

About Paddle's main work related to embedded engineering, after two weeks of discussion and research with @Xreki .

Mainly contains four parts:

• Build Paddle for the embedded(mobile) device.
  This part of the work has been completed 70%. Next, we will continue to improve the Paddle cmake for the embedded environment. Include build Paddle for ios. And build minimum size image for the mobile environment.
• Deploy Paddle to the embedded(mobile) device.
  Currently, on the mobile side, we mainly solve the problem of inference. Now Paddle capi is ready, and we have investigated the capi-based alexnet/vgg/resnet and other models inference implementation. The next step is to get some performance/memory benchmarks. As well as the real application model's inference implementation and performance analysis.
• Paddle Performance
  We estimate that the biggest problem on inference is the lack of performance. At present, there are already some deep optimization libraries(like NNPACK) for the deep learning algorithm(like cnn) on ARM. So, we need to investigate the performance of these libraries, and the method of import these libraries into Paddle. Also, we need to optimize some of the PADDLE code for ARM. In addition, there is a known problem that Paddle needs to optimize memory usage during inference.
• Application
  We have already started trying to apply Paddle to the face and ocr mobile inference. Subsequent problems and progress will also be updated here.

[hedaoyuan]

Build Paddle for the embedded(mobile) device.

• Support the compilation of mainstream embedded devices.
• Meet the embedded environment-related compilation requirements.

TODO

• The documentation describes how to build.
• Build in the NVIDIA DRIVE PX2 environment.
• Extend the pruning capabilities of paddle

DONE

• Build in the Android environment.
• Build in the Raspberry Pi environment.

[hedaoyuan]

Paddle Performance

• Refactor the convolution-related code Refactor the convolution-related code #2196
• Performance optimization for convolution calculation Need to add some new algorithm(like direct, winograd) for convolution calculation. #2177
• This document describes paddle's inference optimization process for mobilenet.

[hedaoyuan]

Close this issue, due to we have a new to-do list. #5782