Skip to content

tiovxmosaic

Jump to bottom
Marco Herrera edited this page Jun 15, 2022 · 20 revisions

TIOVX Mosaic

tiovxmosaic Displays multiple input sources in a mosaic pattern using the TIOVX Modules API.

Hierarchy

GObject
 ╰──GInitiallyUnowned
     ╰──GstObject
         ╰──GstElement
             ╰──GstAggregator
                 ╰──GstTIOVXMiso
                     ╰──GstTIOVXMosaic

Pad templates

src

      video/x-raw
                 format: { (string)NV12, (string)GRAY8, (string)GRAY16_LE }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:batched)
                 format: { (string)NV12, (string)GRAY8, (string)GRAY16_LE }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
           num-channels: [ 1, 16 ]

Availability – always

Direction – src

Object type – GstTIOVXMisoPad

Pad properties:

  • emit-signals: Send signals to signal data consumption

    flags: readable, writable

     Boolean.

     Default: false

  • pool-size: Pool size of the internal buffer pool

    flags: readable, writable, controllable

     Integer.

     Range: 2 - 16

     Default: 2

  • repeat-after-eos: Flag to indicate if the pad will repeat the last buffer after an EOS is received. Only valid for sink pads

    flags: readable, writable, controllable

     Boolean.

     Default: true

sink_%u

Capabilities:

      video/x-raw
                 format: { (string)NV12, (string)GRAY8, (string)GRAY16_LE }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:batched)
                 format: { (string)NV12, (string)GRAY8, (string)GRAY16_LE }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
           num-channels: [ 1, 16 ]

Availability – On Request

Direction – sink

Object type – GstTIOVXMosaicPad

Pad Properties

  • channels: Array of channels to be displayed from this pad. If none are specified these are assumed to be <0, 1, 2, ...>. This is required when using the mosaic pad with batched inputs, in order to filter channels to use. Usage example: <0, 0, 2>

     flags: readable, writable

     GstValueArray of GValues of type "gint".

     Default: <0, 1, 2, ...>.

  • emit-signals: Send signals to signal data consumption

    flags: readable, writable

     Boolean.

     Default: false

  • heights: Heights for each of the enabled channels. Cannot be smaller than 1/4 of the input height or larger than the input height. Default to the input height. This is required when using the mosaic pad with batched inputs, in order to specify the height for the corresponding channel. Usage example: <240, 240, 120>

     Flags : Read / Write

     GstValueArray of GValues of type "gint"

     Default value : Input height

  • pool-size: Pool size of the internal buffer pool

     Flags : Read / Write / Controllable

     Integer. Range: 2 - 16

     Default value : 2

  • repeat-after-eos: Flag to indicate if the pad will repeat the last buffer after an EOS is received. Only valid for sink pads

    flags: readable, writable, controllable

     Boolean.

     Default: true

  • startx: Array of X start of each window for this pad. Windows affecting this pad are determined by the channels property. Usage example: <0, 320, 640>

     Flags : Read / Write

     GstValueArray of GValues of type "gint"

     Default value : 0

  • starty: Array of Y start of each window for this pad. Windows affecting this pad are determined by the channels property. Usage example: <0, 240, 480>

     Flags : Read / Write

     GstValueArray of GValues of type "gint"

     Default value : 0

  • widths: Widths for each of the enabled channels. Cannot be smaller than 1/4 of the input width or larger than the input width. Defaults to the input width. This is required when using the mosaic pad with batched inputs, in order to specify the width for the corresponding channel. Usage example: <320, 320, 160>

     Flags : Read / Write

     GstValueArray of GValues of type "gint"

     Default value : Input width

background

      video/x-raw
                 format: { (string)NV12, (string)GRAY8, (string)GRAY16_LE }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:batched)
                 format: { (string)NV12, (string)GRAY8, (string)GRAY16_LE }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
           num-channels: [ 1, 16 ]

Availability – On request

Direction – sink

Object type – GstTIOVXMisoPad

Pad properties:

  • emit-signals: Send signals to signal data consumption

    flags: readable, writable

     Boolean.

     Default: false

  • pool-size: Pool size of the internal buffer pool

    flags: readable, writable, controllable

     Integer.

     Range: 2 - 16

     Default: 2

  • repeat-after-eos: Flag to indicate if the pad will repeat the last buffer after an EOS is received. Only valid for sink pads

    flags: readable, writable, controllable

     Boolean.

     Default: true

Properties

background

Background image of the Mosaic to be used by this element. It accepts a RAW generated file you can pass through this property. Note: When passing the background image through pad and through property at the same time, the pad one will overlap.

Flags : readable, writable, changeable only in NULL or READY state

String

Default value : ""

latency

Additional latency in live mode to allow upstream to take longer to produce buffers for the current position (in nanoseconds)

Flags : Read / Write

Unsigned Integer64. Range: 2 - 18446744073709551615

Default value : 0

min-upstream-latency

When sources with a higher latency are expected to be plugged in dynamically after the aggregator has started playing, this allows overriding the minimum latency reported by the initial source(s). This is only taken into account when larger than the actually reported minimum latency. (nanoseconds).

Flags : Read / Write

Unsigned Integer64. Range: 2 - 18446744073709551615

Default value : 0

name

The name of the object

Flags : Read / Write

String

Default value : "tiovxmosaic0"

parent

The parent of the object

Flags : Read / Write

GstObject

Default value : N/A

start-time

Start time to use if start-time-selection=set

Flags : Read / Write

Unsigned Integer64. Range: 2 - 18446744073709551615

Default value : 18446744073709551615

start-time-selection

Decides which start time is output

Flags : Read / Write

Enum

Default value : 0 ("zero")

Enum "GstAggregatorStartTimeSelection

(0): zero - Start at 0 running time (default)

(1): first - Start at first observed input running time

(2): set - Set start time with start-time property

target

TIOVX target to use by this element

Flags : Read / Write / Controllable

Enum

Default value : 2, ("VPAC_MSC1_AND_MSC2")

Enum "GstTIOVXMosaicTarget

(0): VPAC_MSC1 - VPAC MSC1

(1): VPAC_MSC2 - VPAC MSC2

(2): VPAC_MSC1_AND_MSC2 - VPAC MSC1 and VPAC MSC2

Pipeline examples

Single input

gst-launch-1.0                                                                               \
videotestsrc is-live=true num-buffers=5 ! "video/x-raw,format=NV12,width=1920,height=1080" ! \
tiovxmosaic ! filesink location=mosaic_out_nv12_1_input.yuv -e -v

Multiple inputs

gst-launch-1.0                                                                                                      \
videotestsrc num-buffers=10 pattern=ball ! "video/x-raw,format=NV12, width=640, height=480" ! queue ! mosaic.sink_0 \
videotestsrc num-buffers=10 pattern=ball ! "video/x-raw,format=NV12, width=640, height=480" ! queue ! mosaic.sink_1 \
videotestsrc num-buffers=10 pattern=ball ! "video/x-raw,format=NV12, width=640, height=480" ! queue ! mosaic.sink_2 \
videotestsrc num-buffers=10 pattern=ball ! "video/x-raw,format=NV12, width=640, height=480" ! queue ! mosaic.sink_3 \
tiovxmosaic name=mosaic                                                                                             \
sink_0::startx="<0>" sink_0::starty="<0>"                                                                           \
sink_1::startx="<640>" sink_1::starty="<480>"                                                                       \
sink_2::startx="<0>" sink_2::starty="<480>"                                                                         \
sink_3::startx="<640>" sink_3::starty="<0>" !                                                                       \
filesink location=mosaic_out_nv12_4_input.yuv -e -v

Using a raw image for background and passing it as background property

BACKGROUND_FILE=background.raw

# Create a background image
gst-launch-1.0  videotestsrc num-buffers=1 is-live=true ! video/x-raw,width=640,height=480,format=NV12 ! filesink location=${BACKGROUND_FILE}

gst-launch-1.0 videotestsrc num-buffers=10 pattern=ball ! video/x-raw,width=320,height=240 ! \
tiovxmosaic name=mosaic background=${BACKGROUND_FILE} !                                                    \
video/x-raw,width=640,height=480,format=NV12 !                                                           \
filesink location=mosaic_out_nv12_background.yuv -e -v

Batched processing

gst-launch-1.0                                                                                                      \
videotestsrc is-live=true num-buffers=10 pattern=ball  ! "video/x-raw, width=640, height=480, format=NV12" ! mux_a. \
videotestsrc is-live=true num-buffers=10 pattern=smpte ! "video/x-raw, width=640, height=480, format=NV12" ! mux_a. \
tiovxmux name=mux_a ! "video/x-raw(memory:batched), width=640, height=480, format=NV12" ! queue ! mosaic.sink_0     \
videotestsrc is-live=true num-buffers=10 pattern=ball  ! "video/x-raw, width=640, height=480, format=NV12" ! mux_b. \
videotestsrc is-live=true num-buffers=10 pattern=smpte ! "video/x-raw, width=640, height=480, format=NV12" ! mux_b. \
tiovxmux name=mux_b ! "video/x-raw(memory:batched), width=640, height=480, format=NV12" ! queue ! mosaic.sink_1     \
tiovxmosaic name=mosaic                                                                                             \
sink_0::channels="<0>"   sink_0::startx="<0>"     sink_0::starty="<0>"                                              \
sink_0::channels="<0,1>" sink_1::startx="<0,640>" sink_1::starty="<480,480>" !                                      \
queue ! filesink location=mosaic_out_nv12_3_channels.yuv -e -v
Clone this wiki locally