Marker op for profiling

[FeixLiu]

PR types

New features

PR changes

OPs

Describe

marker op for profiling the forward and backward steps.

for now, these ops don't have api and should be inserted manually before train steps with these following codes:

            global_block = train_program.global_block() 
            global_block._insert_op(
                index=1,
                type='marker',
                outputs={'Out': temp_var},
                attrs={
                    'marker_role': 'forward',
                    'marker_pos': 'B',
                    'op_role': OpRole.Forward})
            offset = 0
            should_insert_forward_e_marker = True
            should_insert_backward_b_marker = True
            should_insert_backward_e_marker = True
            for idx, op in enumerate(list(global_block.ops)):
                if int(op.attr('op_role')) != int(OpRole.Forward) and int(op.attr('op_role')) != int(
                        OpRole.LRSched) and should_insert_forward_e_marker:
                    global_block._insert_op(
                        index=idx + offset,
                        type='marker',
                        outputs={'Out': temp_var},
                        attrs={
                            'marker_role': 'forward',
                            'marker_pos': 'E',
                            'op_role': OpRole.Forward})
                    should_insert_forward_e_marker = False
                    offset += 1
                if int(op.attr('op_role')) == int(OpRole.Backward) and should_insert_backward_b_marker:
                    global_block._insert_op(
                        index=idx + offset,
                        type='marker',
                        outputs={'Out': temp_var},
                        attrs={
                            'marker_role': 'backward',
                            'marker_pos': 'B',
                            'op_role': OpRole.Backward})
                    should_insert_backward_b_marker = False
                    offset += 1
                if int(op.attr('op_role')) == int(OpRole.Optimize) and should_insert_backward_e_marker:
                    global_block._insert_op(
                        index=idx + offset,
                        type='marker',
                        outputs={'Out': temp_var},
                        attrs={
                            'marker_role': 'backward',
                            'marker_pos': 'E',
                            'op_role': OpRole.Backward})
                    should_insert_backward_e_marker = False

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[gongweibao]

LGTM