add use_mkldnn attribute to ops in dygraph #25773

sfraczek · 2020-07-28T11:05:09Z

PR types

New features

PR changes

APIs

Describe

Adds use_mkldnn argument to ops, similarly to how use_cudnn is used. In Dygraph mode, setting global FLAGS_use_mkldnn=true enables mkldnn for all supported operators. If it is desired to have non-mkldnn op when FLAGS_use_mkldnn=true, it can be done by passing use_mkldnn=False to Op as argument.

paddle-bot-old · 2020-07-28T11:05:14Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

sfraczek · 2020-07-28T13:50:38Z

python/paddle/fluid/layers/nn.py

@@ -11322,7 +11322,7 @@ def scale(x, scale=1.0, bias=0.0, bias_after_scale=True, act=None, name=None):
    return helper.append_activation(out)


-def elementwise_add(x, y, axis=-1, act=None, name=None):
+def elementwise_add(x, y, axis=-1, act=None, name=None, use_mkldnn=False):


The fate of this line shall be decided after we resolve the issue discussed here: #25713 (comment)

python/paddle/fluid/dygraph/nn.py

grygielski · 2020-08-06T12:20:50Z

@luotao1 As suggested here (#25713 (comment)), I've changed a way oneDNN is called from .../layers/nn.py/ to use FLAGS_use_mkldnn. Please let me know if you are fine with this solution.

grygielski · 2020-08-07T08:08:57Z

@luotao1 Could you send me logs of MAC_PR_CI_python35 or rerun if it's a random fail?

luotao1 · 2020-08-07T09:49:43Z

[12:40:59][Step 1/1] 697/904 Test #700: test_slice_var .......................................................***Failed    2.98 sec
[12:40:59][Step 1/1] test_slice_var failed
[12:40:59][Step 1/1]  .E
[12:40:59][Step 1/1] ======================================================================
[12:40:59][Step 1/1] ERROR: test_check_output_8k (test_slice_var.TestSliceVar)
[12:40:59][Step 1/1] ----------------------------------------------------------------------
[12:40:59][Step 1/1] Traceback (most recent call last):
[12:40:59][Step 1/1]   File "/home/teamcity/work/37d0fbdf11d96284/build/python/paddle/fluid/tests/unittests/test_slice_var.py", line 61, in test_check_output_8k
[12:40:59][Step 1/1]     self.check_slice_output(shapes, expected_sizes, 8192)
[12:40:59][Step 1/1]   File "/home/teamcity/work/37d0fbdf11d96284/build/python/paddle/fluid/tests/unittests/test_slice_var.py", line 33, in check_slice_output
[12:40:59][Step 1/1]     shape=shape)
[12:40:59][Step 1/1]   File "/home/teamcity/work/37d0fbdf11d96284/build/python/paddle/fluid/framework.py", line 2640, in create_var
[12:40:59][Step 1/1]     var = Variable(block=self, *args, **kwargs)
[12:40:59][Step 1/1]   File "/home/teamcity/work/37d0fbdf11d96284/build/python/paddle/fluid/framework.py", line 930, in __init__
[12:40:59][Step 1/1]     "matched.".format(self.name, old_shape, shape))
[12:40:59][Step 1/1] ValueError: Variable 45858 has been created before. the previous shape is (8, 1020); the new shape is (800, 10). They are not matched.
[12:40:59][Step 1/1] 
[12:40:59][Step 1/1] ----------------------------------------------------------------------
[12:40:59][Step 1/1] Ran 2 tests in 0.003s
[12:40:59][Step 1/1] 
[12:40:59][Step 1/1] FAILED (errors=1)

It seems not random fail, but I rerun it again for you.

lelelelelez · 2020-08-07T09:53:55Z

@grygielski you can download this log by https://paddle-docker-tar.cdn.bcebos.com/buildLog/386139.log

jczaja

LGTM

luotao1 · 2020-08-11T08:19:33Z

2020-08-06 20:00:32 
2020-08-06 20:00:34 ****************
2020-08-06 20:00:34 0. You must have one RD (XiaoguangHu01 or lanxianghit) and one TPM (saxon-zh or jzhang533 or swtkiwi or Heeenrrry or TCChenlong) approval for the api change for the management reason of API interface.
2020-08-06 20:00:34 1. You must have one TPM (saxon-zh or jzhang533 or swtkiwi or Heeenrrry or TCChenlong) approval for the api change for the management reason of API document.
2020-08-06 20:00:34 
2020-08-06 20:00:34 There are 2 approved errors.
2020-08-06 20:00:34 ****************
2020-08-06 20:00:34 API Difference is: 
2020-08-06 20:00:34 - paddle.fluid.dygraph.Conv2D (paddle.fluid.dygraph.nn.Conv2D, ('document', 'eaf7f009cbb4ce2f03453b767b8e3286'))
2020-08-06 20:00:34 ?                                                                            ^^   ^^ --    ^ ----- ^^^^ -----
2020-08-06 20:00:34 
2020-08-06 20:00:34 + paddle.fluid.dygraph.Conv2D (paddle.fluid.dygraph.nn.Conv2D, ('document', 'c1fc780ffd5769b4c16e71608c77fbf8'))
2020-08-06 20:00:34 ?                                                                            ^^ + ++ ^^^^^    ++ ^^^^^^^^  ^
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.fluid.dygraph.Conv2D.__init__ (ArgSpec(args=['self', 'num_channels', 'num_filters', 'filter_size', 'stride', 'padding', 'dilation', 'groups', 'param_attr', 'bias_attr', 'use_cudnn', 'act', 'dtype'], varargs=None, keywords=None, defaults=(1, 0, 1, None, None, None, True, None, 'float32')), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 + paddle.fluid.dygraph.Conv2D.__init__ (ArgSpec(args=['self', 'num_channels', 'num_filters', 'filter_size', 'stride', 'padding', 'dilation', 'groups', 'param_attr', 'bias_attr', 'use_cudnn', 'use_mkldnn', 'act', 'dtype'], varargs=None, keywords=None, defaults=(1, 0, 1, None, None, None, True, True, None, 'float32')), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 ?                                                                                                                                                                                               ++++++++++++++                                                                               ++++++
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.fluid.dygraph.Pool2D (paddle.fluid.dygraph.nn.Pool2D, ('document', '1a697b10e08b443dc2865bfa1e1422a5'))
2020-08-06 20:00:34 ?                                                                               -----  ^^  ^ ----------------
2020-08-06 20:00:34 
2020-08-06 20:00:34 + paddle.fluid.dygraph.Pool2D (paddle.fluid.dygraph.nn.Pool2D, ('document', '28cc18bbe69ffb12e449a6e0344baebd'))
2020-08-06 20:00:34 ?                                                                            ++++ +++++++++++++++    ^  ^^^^
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.fluid.dygraph.Pool2D.__init__ (ArgSpec(args=['self', 'pool_size', 'pool_type', 'pool_stride', 'pool_padding', 'global_pooling', 'use_cudnn', 'ceil_mode', 'exclusive', 'data_format'], varargs=None, keywords=None, defaults=(-1, 'max', 1, 0, False, True, False, True, 'NCHW')), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 + paddle.fluid.dygraph.Pool2D.__init__ (ArgSpec(args=['self', 'pool_size', 'pool_type', 'pool_stride', 'pool_padding', 'global_pooling', 'use_cudnn', 'use_mkldnn', 'ceil_mode', 'exclusive', 'data_format'], varargs=None, keywords=None, defaults=(-1, 'max', 1, 0, False, True, True, False, True, 'NCHW')), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 ?                                                                                                                                               ++++++++++++++                                                                                                            ++++++
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.fluid.dygraph.Linear (paddle.fluid.dygraph.nn.Linear, ('document', 'db61dfab21ff2bd3984cccdfcb559570'))
2020-08-06 20:00:34 ?                                                                            ^^^^^^^^^^^^^^^ --   ^   ^^^ ^^
2020-08-06 20:00:34 
2020-08-06 20:00:34 + paddle.fluid.dygraph.Linear (paddle.fluid.dygraph.nn.Linear, ('document', '84844734ccb96fadf34fbce9f2a8acf0'))
2020-08-06 20:00:34 ?                                                                            ^^^^^^    ^^^^^  ++++ ^ ^^^^^^^
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.fluid.dygraph.Linear.__init__ (ArgSpec(args=['self', 'input_dim', 'output_dim', 'param_attr', 'bias_attr', 'act', 'dtype'], varargs=None, keywords=None, defaults=(None, None, None, 'float32')), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 + paddle.fluid.dygraph.Linear.__init__ (ArgSpec(args=['self', 'input_dim', 'output_dim', 'param_attr', 'bias_attr', 'act', 'dtype', 'use_mkldnn'], varargs=None, keywords=None, defaults=(None, None, None, 'float32', True)), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 ?                                                                                                                                ++++++++++++++                                                                      ++++++
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.fluid.dygraph.BatchNorm (paddle.fluid.dygraph.nn.BatchNorm, ('document', 'e29cbd4c024975a79f1980cebacc4651'))
2020-08-06 20:00:34 ?                                                                                  ^  ^^^^^^^^^^^^^^    ^^ ---- ^^^
2020-08-06 20:00:34 
2020-08-06 20:00:34 + paddle.fluid.dygraph.BatchNorm (paddle.fluid.dygraph.nn.BatchNorm, ('document', '8882593f14f554039a93c664f7f8be43'))
2020-08-06 20:00:34 ?                                                                                  ^^^ + ^  +++++++ ++++++++++ ^  ^
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.fluid.dygraph.BatchNorm.__init__ (ArgSpec(args=['self', 'num_channels', 'act', 'is_test', 'momentum', 'epsilon', 'param_attr', 'bias_attr', 'dtype', 'data_layout', 'in_place', 'moving_mean_name', 'moving_variance_name', 'do_model_average_for_mean_and_var', 'use_global_stats', 'trainable_statistics'], varargs=None, keywords=None, defaults=(None, False, 0.9, 1e-05, None, None, 'float32', 'NCHW', False, None, None, True, False, False)), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 + paddle.fluid.dygraph.BatchNorm.__init__ (ArgSpec(args=['self', 'num_channels', 'act', 'is_test', 'momentum', 'epsilon', 'param_attr', 'bias_attr', 'dtype', 'data_layout', 'in_place', 'moving_mean_name', 'moving_variance_name', 'do_model_average_for_mean_and_var', 'use_global_stats', 'trainable_statistics', 'use_mkldnn'], varargs=None, keywords=None, defaults=(None, False, 0.9, 1e-05, None, None, 'float32', 'NCHW', False, None, None, True, False, False, True)), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 ?                                                                                                                                                                                                                                                                                                                   ++++++++++++++                                                                                                                                      ++++++
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.nn.BatchNorm (paddle.fluid.dygraph.nn.BatchNorm, ('document', 'e29cbd4c024975a79f1980cebacc4651'))
2020-08-06 20:00:34 ?                                                                       ^  ^^^^^^^^^^^^^^    ^^ ---- ^^^
2020-08-06 20:00:34 
2020-08-06 20:00:34 + paddle.nn.BatchNorm (paddle.fluid.dygraph.nn.BatchNorm, ('document', '8882593f14f554039a93c664f7f8be43'))
2020-08-06 20:00:34 ?                                                                       ^^^ + ^  +++++++ ++++++++++ ^  ^
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.nn.BatchNorm.__init__ (ArgSpec(args=['self', 'num_channels', 'act', 'is_test', 'momentum', 'epsilon', 'param_attr', 'bias_attr', 'dtype', 'data_layout', 'in_place', 'moving_mean_name', 'moving_variance_name', 'do_model_average_for_mean_and_var', 'use_global_stats', 'trainable_statistics'], varargs=None, keywords=None, defaults=(None, False, 0.9, 1e-05, None, None, 'float32', 'NCHW', False, None, None, True, False, False)), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 + paddle.nn.BatchNorm.__init__ (ArgSpec(args=['self', 'num_channels', 'act', 'is_test', 'momentum', 'epsilon', 'param_attr', 'bias_attr', 'dtype', 'data_layout', 'in_place', 'moving_mean_name', 'moving_variance_name', 'do_model_average_for_mean_and_var', 'use_global_stats', 'trainable_statistics', 'use_mkldnn'], varargs=None, keywords=None, defaults=(None, False, 0.9, 1e-05, None, None, 'float32', 'NCHW', False, None, None, True, False, False, True)), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 ?                                                                                                                                                                                                                                                                                                       ++++++++++++++                                                                                                                                       ++++++
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.nn.Pool2D (paddle.fluid.dygraph.nn.Pool2D, ('document', '1a697b10e08b443dc2865bfa1e1422a5'))
2020-08-06 20:00:34 ?                                                                    -----  ^^  ^ ----------------
2020-08-06 20:00:34 
2020-08-06 20:00:34 + paddle.nn.Pool2D (paddle.fluid.dygraph.nn.Pool2D, ('document', '28cc18bbe69ffb12e449a6e0344baebd'))
2020-08-06 20:00:34 ?                                                                 ++++ +++++++++++++++    ^  ^^^^
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.nn.Pool2D.__init__ (ArgSpec(args=['self', 'pool_size', 'pool_type', 'pool_stride', 'pool_padding', 'global_pooling', 'use_cudnn', 'ceil_mode', 'exclusive', 'data_format'], varargs=None, keywords=None, defaults=(-1, 'max', 1, 0, False, True, False, True, 'NCHW')), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 + paddle.nn.Pool2D.__init__ (ArgSpec(args=['self', 'pool_size', 'pool_type', 'pool_stride', 'pool_padding', 'global_pooling', 'use_cudnn', 'use_mkldnn', 'ceil_mode', 'exclusive', 'data_format'], varargs=None, keywords=None, defaults=(-1, 'max', 1, 0, False, True, True, False, True, 'NCHW')), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 ?                                                                                                                                    ++++++++++++++                                                                                                            ++++++
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.nn.Linear (paddle.fluid.dygraph.nn.Linear, ('document', 'db61dfab21ff2bd3984cccdfcb559570'))
2020-08-06 20:00:34 ?                                                                 ^^^^^^^^^^^^^^^ --   ^   ^^^ ^^
2020-08-06 20:00:34 
2020-08-06 20:00:34 + paddle.nn.Linear (paddle.fluid.dygraph.nn.Linear, ('document', '84844734ccb96fadf34fbce9f2a8acf0'))
2020-08-06 20:00:34 ?                                                                 ^^^^^^    ^^^^^  ++++ ^ ^^^^^^^
2020-08-06 20:00:34 
2020-08-06 20:00:34 - paddle.nn.Linear.__init__ (ArgSpec(args=['self', 'input_dim', 'output_dim', 'param_attr', 'bias_attr', 'act', 'dtype'], varargs=None, keywords=None, defaults=(None, None, None, 'float32')), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 + paddle.nn.Linear.__init__ (ArgSpec(args=['self', 'input_dim', 'output_dim', 'param_attr', 'bias_attr', 'act', 'dtype', 'use_mkldnn'], varargs=None, keywords=None, defaults=(None, None, None, 'float32', True)), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))
2020-08-06 20:00:34 ?                                                                                                                      ++++++++++++++                                                                     ++++++
2020-08-06 20:00:34 
2020-08-06 20:00:36 + approval_error=1
2020-08-06 20:00:36 + '[' 1 '!=' 0 ']'

luotao1 · 2020-08-11T08:23:31Z

python/paddle/fluid/dygraph/nn.py

@@ -810,6 +831,7 @@ def __init__(self,
                 pool_padding=0,
                 global_pooling=False,
                 use_cudnn=True,
+                 use_mkldnn=True,


Since we use FLAGS_use_mkldnn to global control, do you need to add use_mkldnn attribute in each API?

As far as I understood our agreement, we didn't want to add use_mkldnn attribute in standard op API, thus we have decided to use Flags here. However for dygraph ops we wanted to be able to control every op whether it should be executed with oneDNN or not. Similarly these ops have use_cuda attribute. Am I wrong?

@phlrain What's your opinion about it?

However for dygraph ops we wanted to be able to control every op whether it should be executed with oneDNN or not

@grygielski Discussed with @phlrain, we think FLAGS_use_mkldnn is enough for dygraph, we don't need to control every op whether it should be executed with oneDNN or not

grygielski · 2020-08-17T12:51:21Z

@luotao1 I've removed use_mkldnn attribute from all operators. I didn't delete it from dygraph_utils.py. Let me know if you are fine with that.

luotao1

LGTM

wojtuss

LGTM

$sfraczek$

$@sfraczek$

add use_mkldnn attribute to ops in dygraph

8152153

$@sfraczek$ sfraczek added dygraph issues related to dygraph mode Intel labels Jul 28, 2020

$@sfraczek$ sfraczek requested review from wojtuss, wozna and grygielski July 28, 2020 11:05

$sfraczek$

sfraczek commented Jul 28, 2020

View reviewed changes

$@sfraczek$ sfraczek mentioned this pull request Jul 28, 2020

Added Relu6 mkldnn op #25713

Closed

wojtuss reviewed Jul 29, 2020

View reviewed changes

python/paddle/fluid/dygraph/nn.py Outdated Show resolved Hide resolved

sfraczek added 2 commits July 29, 2020 13:10

$@sfraczek$

review fixes (use_mkl_dnn->use_mkldnn)

58c7f0a

$@sfraczek$

add test for LayerObjectHelper.append_activation

d02209b

$@sfraczek$ sfraczek mentioned this pull request Jul 31, 2020

Verification of dygraph MKLDNN accuracy convergence #25872

Closed

Use FLAGS_use_mkldnn instead of additional parameter

fbab971

grygielski requested review from lidanqing-intel and jczaja August 10, 2020 07:55

jczaja previously approved these changes Aug 10, 2020

View reviewed changes

grygielski assigned luotao1 Aug 10, 2020

luotao1 reviewed Aug 11, 2020

View reviewed changes

grygielski added 3 commits August 17, 2020 03:36

Remove use_mkldnn attributes

086604c

Fix errors in nn.py

4f1cdf7

Lint fixes

d4f34c5

grygielski dismissed jczaja’s stale review via e60183d August 17, 2020 09:14

grygielski force-pushed the add-use_mkldnn-attribute-to-ops-in-dygraph branch 2 times, most recently from 3584eb0 to 0da66fd Compare August 17, 2020 09:22

grygielski force-pushed the add-use_mkldnn-attribute-to-ops-in-dygraph branch 2 times, most recently from 480ed1d to 086604c Compare August 17, 2020 09:33

luotao1 approved these changes Aug 18, 2020

View reviewed changes

wojtuss approved these changes Aug 18, 2020

View reviewed changes

luotao1 merged commit abfdffa into PaddlePaddle:develop Aug 18, 2020

$@sfraczek$ sfraczek deleted the add-use_mkldnn-attribute-to-ops-in-dygraph branch September 1, 2020 08:56

arlesniak mentioned this pull request Nov 3, 2020

Add command values to beginning of log and use paddle.enable_mkldnn()) to instead of FLAGS_use_mkldnn #27935

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add use_mkldnn attribute to ops in dygraph #25773

add use_mkldnn attribute to ops in dygraph #25773

$@sfraczek$ sfraczek commented Jul 28, 2020

paddle-bot-old bot commented Jul 28, 2020

$@sfraczek$ sfraczek Jul 28, 2020

grygielski commented Aug 6, 2020

grygielski commented Aug 7, 2020 •

edited

Loading

luotao1 commented Aug 7, 2020

lelelelelez commented Aug 7, 2020

jczaja left a comment

luotao1 commented Aug 11, 2020

luotao1 Aug 11, 2020

grygielski Aug 11, 2020

luotao1 Aug 11, 2020

luotao1 Aug 11, 2020

grygielski commented Aug 17, 2020

luotao1 left a comment

wojtuss left a comment

add use_mkldnn attribute to ops in dygraph #25773

add use_mkldnn attribute to ops in dygraph #25773

Conversation

sfraczek commented Jul 28, 2020

PR types

PR changes

Describe

paddle-bot-old bot commented Jul 28, 2020

sfraczek Jul 28, 2020

Choose a reason for hiding this comment

grygielski commented Aug 6, 2020

grygielski commented Aug 7, 2020 • edited Loading

luotao1 commented Aug 7, 2020

lelelelelez commented Aug 7, 2020

jczaja left a comment

Choose a reason for hiding this comment

luotao1 commented Aug 11, 2020

luotao1 Aug 11, 2020

Choose a reason for hiding this comment

grygielski Aug 11, 2020

Choose a reason for hiding this comment

luotao1 Aug 11, 2020

Choose a reason for hiding this comment

luotao1 Aug 11, 2020

Choose a reason for hiding this comment

grygielski commented Aug 17, 2020

luotao1 left a comment

Choose a reason for hiding this comment

wojtuss left a comment

Choose a reason for hiding this comment

$@sfraczek$ sfraczek commented Jul 28, 2020

$@sfraczek$ sfraczek Jul 28, 2020

grygielski commented Aug 7, 2020 •

edited

Loading