[MKLDNN] fast_rcnn 单侧挂

[vivienfanghuagood]

bug描述 Describe the Bug

FastRCNN单侧挂掉。

单测：https://github.com/EmmonsCurse/PaddleTest/blob/model_source_update/inference/python_api_test/test_det_model/test_fast_rcnn_mkldnn.py

模型文件：https://paddle-qa.bj.bcebos.com/inference_model_clipped/2.5.1/detection/fast_rcnn.tgz

复现步骤（安装 paddle whl 包后）：
git clone https://github.com/EmmonsCurse/PaddleTest.git -b model_source_update

cd PaddleTest/inference/python_api_test/

python -m pip install -r requirements.txt -i https://mirror.baidu.com/pypi/simple

cd test_det_model

python -m pytest -m server -k mkldnn --disable-warnings -sv test_fast_rcnn_mkldnn.py

其他补充信息 Additional Supplementary Information

No response

[vivienfanghuagood]

[xinyu-intel]

Thanks for reporting the issue, will address soon.