#45898) (#45931)

* Allow manaully set py_reader name in standalone executor

* Fix CI errors

修复cuda11.7编译出错的问题

* cherry pick delay tensorrt log
* Update trt_plugin.h

* Support dynamic shape in multiclass_nms3 Plugin for Paddle-TensorRT.

* fix arm fp16 compile error

* polish macro impl

* delete eigen header in data_type.h

* fix complie bug

* refactor

Support 0 shapes input Tensor for MKL slice kernel

…undefined-var (#45916) (#46020)

* fix select_input with different shape errors:
1. select_input_with_buildin_type directly return non-undefinedvar branch when meeting undefined var
2. the output shape of select_input is inferred from inputs.

* reverse the logic in select_input

* normalize yaml file name (#45894)

* Clear extra attributes of activation op in OpMaker (#45772)

* clear extra attr of activation op in opmaker

* fix syntax bug

* fix mkldnn kernel

* fix merge conflict

* fix bug

* [PHI] Normalize yaml op label (#45976)

* normalize yaml op label

* revert op_compat yaml change

* fix prelu and rnn compat problem

* replace api by op

* support assign op backward refuse forward (#45879)

* normize yaml backward op label (#46028)

Co-authored-by: zyfncg <zhangyunfei07@baidu.com>
Co-authored-by: Charles-hit <56987902+Charles-hit@users.noreply.github.com>

* skip forward save

* fix bug

* more ci for jit skip forward

…s for fill_any_like op (#46079)

* Fix split bug in static mode (#45906)

* fix split bug in static mode

* modify code style

* modify code style

* add unit test for split

* add convert rules for fill_any_like op in paddle science (#45985)

* add convert rules for fill_any_like op in paddle science

* add unit test for fill_any_like op in paddle science

* modify fill_any_like convert rule

* modify fill_any_like convert rule dtype

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fix gloo compile warning

* adapt for nn fl-ps

* unify  core_avx and core_noavx

* fix except error

* revert mac compile logic

* revert dylib to so

* add core_noavx branch

* remove core_noavx

* replace paddle_core by lib paddle

* polish var name

* replace paddle_core by libpaddle

* update custom device commit

* polish code by comments

fix bug for TransformedDistribution

This reverts commit c252b1d.

* make eager log readable

* fix compile error

* recover test

* invoke ci again

* [AutoParallel] adapt gradient merge pass (#45915)

* adapt gradient merge

* fix op_role

* fix strategy

* [Auto Parallel] Gradient Fuse Allreduce (#45643)

* bugfix (#45332)

* dist embedding support lookup table v1

* add unitest

* customize wait_comm

* group gradients

* bugfix

* update program

* [Auto Parallel] Improve the APIs (#45776)

* [Auto Parallel] Use c++ dist attr in the completion process

* [Auto Parallel] Add minor changes

* [Auto Parallel] Use c++ dist attr in the completion process

* [Auto Parallel] Add minor changes

* [Auto Parallel] Add the serialization process for dist attrs

* [Auto Parallel] Remove unnecessary comments

* [Auto Parallel] Fix some bugs

* [Auto Parallel] Fix the code style

* [Auto Parallel] Remove unnecessary impls

* [Auto Parallel] Fix the importing error

* [Auto Parallel] Fix the copy from bugs of op dist attr

* [Auto Parallel] Replace the use of constexpr if

* [Auto Parallel] Redesign the shard_tensor, shard_op and ProcessMesh

* [Auto Parallel] Change API of the completion unittest

* [Auto Parallel] Fix the bug when set_attr an int

* [Auto Parallel] Add the unittest for the serialization

* [Auto Parallel] Add some unit tests

* [Auto Paralle] Unify the strategy

* [Auto Parallel] Improve the engine api

* [Auto Parallel] Reset the changes made to the framework

* [Auto Parallel] Change the engine unittest

* [Auto Parallel] Update API of the completion and partitioner

* [Auto Parallel] Update unit tests using engine api

* update shard annotation

* [Auto Parallel] Remove the modifications of other modules

* [Auto Parallel] Add docs for APIs

* add new strategy

* [Auto Parallel] Replace the logger

* [Auto Parallel] Restore the test_program.py

* [Auto Parallel] Change the import rules

* [Auto Parallel] Add the examples for Engine

* [Auto Parallel] Do some minor changes

* [Auto Parallel] Remove yaml dependency

* [Auto Parallel] Fix the unittests

* add valid after train

* bug fix

Co-authored-by: zhaoyingli <zhaoyingli@baidu.com>
Co-authored-by: caozhou <caozhou@radi.ac.cn>
Co-authored-by: caozhou <48191911+Caozhou1995@users.noreply.github.com>

* [Auto Parallel] Bugfix allreduce fuse for MP (#46086)

* bugfix

* bugfix

* typos fixed

* update strategy (#46138)

Co-authored-by: zhaoyingli <86812880+zhaoyinglia@users.noreply.github.com>
Co-authored-by: JZ-LIANG <jianzhongliang10@gmail.com>
Co-authored-by: zhaoyingli <zhaoyingli@baidu.com>
Co-authored-by: caozhou <caozhou@radi.ac.cn>
Co-authored-by: caozhou <48191911+Caozhou1995@users.noreply.github.com>

* refactor mp.

* update setup.py.

* update mp_layers.py for compatibility.

* add documents for mp_layers.py

* update init.py

* update collective.py.

* update.

* update mp_ops.py

* update.

* update code style.

* update code style.

Co-authored-by: RichardWooSJTU <37864677+RichardWooSJTU@users.noreply.github.com>

* [PHI] Support bmm and bmm_grad in xpu (#45887)

* support bmm and bmm_grad in xpu

* add error removal

* test=kunlun

* refactor code for better structure

* test=kunlun

* add fp16 kernel for bmm

* test=kunlun

* test=kunlun

* add unit test for sum higher level op (#45961)

* support slice op backward refuse forward and add high level unit test (#45960)

* support tile op backward refuse forward (#45942)

* support expand_v2 op backward refuse forward (#45941)

* support concat backward refuse forward (#45940)

…46179)

…ram (#46194)

* [dy2static] support user to use decorator in their program (#45768)

* support deco

* fix deco ast type

* arg_str

* 1

* support callable deco

* code style

* codestyle

* test_error

* fix decos in another file

* recover conflict codes

* [BugFix] fixed a bug in decorator transformer, it can not analyze decorator with params correctly (#46055)

* fix deco call

* add raise

* add test

* add warn, fix paddle api

* fix error type

* fix coverage

…tion operators (#46184)

* [cherry-pick] extend reduce_sum,reduce_sum,eq,ne,ge,abs,pow,etc higher order operators

* add reduce_mean,reduce_sum primitive ops
* add ne_p gt_p primitive operators
* add ge_p abs_p primitive oparators
* add cast primitive operators
* add pow,square prim2oirg rules
* add elementwise_div orig2prim rule

* [cherry-pick] add mean,sum,ge,gt,ne,abs,etc higher-order differentiation operators(#45888)

* add reduce_mean,reduce_sum primitive ops

* add ne_p gt_p primitive operators

* add ge_p abs_p primitive oparators

…ecific inputs (#46148) (#46193)

* fix return order error and duplicate results with specific inputs

* fix wrong eigen header include

* fix complie bug

* fix nan_inf_utils_detail

* fix resource_manager

* fix conv_miopen_helper

* fix static_check error when compile twice (#46140)

* [CI] fix static check in build_pr_dev (#46192)

Co-authored-by: Zhou Wei <1183042833@qq.com>

cherry-pick from #45826
LayoutAutotune 支持 inplace 类型的OP
 根据 Add eager layout autotune #45409 修改意见调整UseAutotune
将LayoutAutotune判断放到controller中，与AMP 判断保持一致

* add scope cache & reuse

* add gc scope for end of each train step

* del scope reuse for jit

* refine code

* test

…46178) (#46260)

* fix preln_residual_bias_fuse_pass bug in TNT_small model

* support cast op backward refuse forward and fix some bugs (#46173)

* support cast op backward refuse forward

* Fix the bug of high order unit test framework

* support sign op backward refuse forward (#46002)

* fix linspace error in amp

* fix log

* fix amp error

cherry-pick : #46016, #46021, #45974

* [Sparse]Sparse add support gpu (#45974)

* [Sparse]Remove unused code (#46021)

* [Sparse] Add infer meta (#46016)

…) (#46186)

* Fix TransDataBackend Error when call unsqueeze using MKL Tensor

* Add UT

* Refine UT

* add config

* add config

* follow comments

* fix serial run

* Support matmul_v2 in Paddle-TensorRT converter.

* Fix bug of reduce_sum op. When input.numel() > INT32_MAX,
its result is wrong.

* Cherry-pick of PR 46045

* Fix bug of reduce_sum kp op.

* Fix bug of reduce_sum kp operator compilation.
If compilation device is XPU, eigen kernel should be ignored.

* [Eager] Fix ocr (#46124)

* fix linspace error in amp

* fix log

* fix amp error

* fix ocr error which caused by amp

* add more check

* rename dtype ns

* [Eager Bug fix]Fix Detection (#46147)

* fix linspace error in amp

* fix log

* fix amp error

* Revert "Simplify size op impl (#45808)"

This reverts commit c252b1d.

* fix_seg

* fix detection

Co-authored-by: Chen Weihang <sunny_cwh@163.com>

Co-authored-by: Chen Weihang <sunny_cwh@163.com>

* [Auto Parallel] Change the import way of Auto Parallel (#46115)

* fix strategy (#46256)

* [Auto Parallel] performance improvement for Sharding-DP hybrid parallelism (#46180)

* remove no need grad allreduce communication when sharding-dp

* remove no need grad allreduce communication when sharding-dp

* bugfix

* bugfix

* bugfix

Co-authored-by: Yulong Ao <aoyulong@baidu.com>
Co-authored-by: JZ-LIANG <jianzhongliang10@gmail.com>

* polish code comments

* polish data_device_transform.cc

…6280)

* Move ITensor construction for Weight (persistable variable) from OpConvert to TensorRTEngine.

* [XPU] update xdnn activations. (#46246)

* [XPU] update xpu cmake. test=kunlun

* fix cast bug

This reverts commit a2b2af9.

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fix gloo compile warning

* adapt for nn fl-ps

* flps del fake-init op

* add learning_rate_0 intializer op

Co-authored-by: Zhou Wei <1183042833@qq.com>

…46273)

* [JitLayer]Erase out vars to avoid data rewrittinig

* Fix code comments

…nsor] (#46360)

* [Check]Enhance pooling output_size type check

* add unittest

* fix bugs of mp

* fix bugs of mp

* update

* update

* fix bug

uniform logger manager in FleetAPI.
hidde API under distributed/utils which users don't need.

…ershape bug (#46409)

* [BugFix]Fix reduce_mean/min/sum/prod, cumsum grad_op infershape bug

* fix typo

* fix typo

#46366)

* use re replace judge by case

* simplify re

* unexpose audio ParameterError

* clean audio utils api

* fix sub sign reverse for mkldnn

* refactor code as comment

* remove useless

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fix gloo compile warning

* adapt for nn fl-ps

* flps del fake-init op

* add learning_rate_0 intializer op

* bug fix

* .

* .

… 46060) (#46218)

* Clear extra attrs of elementwise op in OpMaker (#45845)

* clear extra attrs of elementwise op in opmaker

* fix op_debug_string_test

* fix bug of grad_add

* fix sort of runtime attrs

* Clear extra attrs of scale in OpMaker (#45984)

* clear extra attr of scale in opmaker

* fix sum bug

* fix merge conflict

* fix minus

* Clear extra attributes of some Op in OpMaker (Part4) (#46060)

* clear extra attr of some ops in opmaker

* revert clear use_cudnn for pool

* fix test_operator_desc

* fix Attr interface of OperatorBase

* fix code stype

#46418, #46451, #46457) (#46553)

* Clear extra attributes of some Op in OpMaker (Part4) (#46060)

* clear extra attr of some ops in opmaker

* revert clear use_cudnn for pool

* fix test_operator_desc

* fix Attr interface of OperatorBase

* clear extra attrs of condition op in opmaker (#46150)

* Clear extra attrs of lookup_table_v2 in OpMaker (#46321)

* clear extra attrs of look_up_table_v2 in opmaker

* fix bug

* clear extra attrs of quantize op in opmaker (#46418)

* delete repeated item

* clear extra attrs of distribute op in opmaker (#46451)

* clear extra atts of sequence_softmax in opmaker (#46457)

* fix libpaddle soname mismatch error

* fix windows failed

* polish linux and windows make impl

* unify winddows lib name

* fix windows error

* revert copy dst change

* revert naming change

* revert windows change

* fix gpups compile failed

…seFloorDivideFunctor (#45051) (#46504)

* set flag of clip_extra in save_inference_model to true (#46151)

* open the clip_extra flag in paddle.static.save_inference_model, test=allcase (#46456)

* Open the clip_extra flag in TracedLayer.save_inference_model (#46473)

* open the clip_extra flag in paddle.static.save_inference_model, test=allcase

* set the defalut value of clip_extra in TracedLayer from False to True, test=allcase

* update english doc of paddle.static.save_inference_model, test=document_fix (#46484)

* Fix clip_extra logic in remove_training_info (#46534)

* fix clip_extra code in remove_training_info

* revert rnn opmaker clear

…GPU (#46641)

Add FP16 support for uniform in dygraph mode on Nvidia GPU
Dev PR link PR46212