forked from GStreamer/gst-plugins-good
-
Notifications
You must be signed in to change notification settings - Fork 0
/
NEWS
2204 lines (1675 loc) · 94.8 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
GStreamer 1.18 Release Notes
GStreamer 1.18.0 was originally released on 7 September 2020.
See https://gstreamer.freedesktop.org/releases/1.18/ for the latest
version of this document.
Last updated: Monday 7 September 2020, 10:30 UTC (log)
Introduction
The GStreamer team is proud to announce a new major feature release in
the stable 1.x API series of your favourite cross-platform multimedia
framework!
As always, this release is again packed with many new features, bug
fixes and other improvements.
Highlights
- GstTranscoder: new high level API for applications to transcode
media files from one format to another
- High Dynamic Range (HDR) video information representation and
signalling enhancements
- Instant playback rate change support
- Active Format Description (AFD) and Bar Data support
- ONVIF trick modes support in both GStreamer RTSP server and client
- Hardware-accelerated video decoding on Windows via DXVA2 /
Direct3D11
- Microsoft Media Foundation plugin for video capture and
hardware-accelerated video encoding on Windows
- qmlgloverlay: New overlay element that renders a QtQuick scene over
the top of an input video stream
- New imagesequencesrc element to easily create a video stream from a
sequence of jpeg or png images
- dashsink: Add new sink to produce DASH content
- dvbsubenc: DVB Subtitle encoder element
- TV broadcast compliant MPEG-TS muxing with constant bitrate muxing
and SCTE-35 support
- rtmp2: new RTMP client source and sink element implementation
- svthevcenc: new SVT-HEVC-based H.265 video encoder
- vaapioverlay compositor element using VA-API
- rtpmanager support for Google’s Transport-Wide Congestion Control
(twcc) RTP extension
- splitmuxsink and splitmuxsrc gained support for auxiliary video
streams
- webrtcbin now contains some initial support for renegotiation
involving stream addition and removal
- New RTP source and sink elements to easily set up RTP streaming via
rtp:// URIs
- New Audio Video Transport Protocol (AVTP) plugin for Time-Sensitive
Applications
- Support for the Video Services Forum’s Reliable Internet Stream
Transport (RIST) TR-06-1 Simple Profile
- Universal Windows Platform (UWP) support
- rpicamsrc element for capturing from the Raspberry Pi camera
- RTSP Server TCP interleaved backpressure handling improvements as
well as support for Scale/Speed headers
- GStreamer Editing Services gained support for nested timelines,
per-clip speed rate control and the OpenTimelineIO format.
- Autotools build system has been removed in favour of Meson
Major new features and changes
Noteworthy new features and API
Instant playback rate changes
Changing the playback rate as quickly as possible so far always required
a flushing seek. This generally works, but has the disadvantage of
flushing all data from the playback pipeline and requiring the demuxer
or parser to do a full-blown seek including resetting its internal state
and resetting the position of the data source. It might also require
considerable decoding effort to get to the right position to resume
playback from at the higher rate.
This release adds a new mechanism to achieve quasi-instant rate changes
in certain playback pipelines without interrupting the flow of data in
the pipeline. This is activated by sending a seek with the
GST_SEEK_FLAG_INSTANT_RATE_CHANGE flag and start_type = stop_type =
GST_SEEK_TYPE_NONE. This flag does not work for all pipelines, in which
case it is necessary to fall back to sending a full flushing seek to
change the playback rate. When using this flag, the seek event is only
allowed to change the current rate and can modify the trickmode flags
(e.g. keyframe only or not), but it is not possible to change the
current playback position, playback direction or do a flush.
This is particularly useful for streaming use cases like HLS or DASH
where the streaming download should not be interrupted when changing
rate.
Instant rate changing is handled in the pipeline in a specific sequence
which is detailed in the seeking design docs. Most elements don’t need
to worry about this, only elements that sync to the clock need some
special handling which is implemented in the GstBaseSink base class, so
should be taken care of automatically in most normal playback pipelines
and sink elements.
See Jan’s GStreamer Conference 2019 talk “Changing Playback Rate
Instantly” for more information.
You can try this feature by passing the -i command line option to
gst-play-1.0. It is supported at least by qtdemux, tsdemux, hlsdemux,
and dashdemux.
Google Transport-Wide Congestion Control
rtpmanager now supports the parsing and generating of RTCP messages for
the Google Transport-Wide Congestion Control RTP Extension, as described
in:
https://tools.ietf.org/html/draft-holmer-rmcat-transport-wide-cc-extensions-01.
This “just” provides the required plumbing/infrastructure, it does not
actually make effect any actual congestion control on the sender side,
but rather provides information for applications to use to make such
decisions.
See Håvard’s “Google Transport-Wide Congestion Control” talk for more
information about this feature.
GstTranscoder: a new high-level transcoding API for applications
The new GstTranscoder library, along with transcodebin and
uritranscodebin elements, provides high level API for applications to
transcode media files from one format to another. Watch Thibault’s talk
“GstTranscoder: A High Level API to Quickly Implement Transcoding
Capabilities in your Applications” for more information.
This also comes with a gst-transcoder-1.0 command line utility to
transcode one URI into another URI based on the specified encoding
profile.
Active Format Description (AFD) and Bar Data support
The GstVideo Ancillary Data API has gained support for Active Format
Description (AFD) and Bar data.
This includes various two new buffer metas: GstVideoAFDMeta and
GstVideoBarMeta.
GStreamer now also parses and extracts AFD/Bar data in the h264/h265
video parsers, and supports both capturing them and outputting them in
the decklink elements. See Aaron’s lightning talk at the GStreamer
Conference for more background.
ONVIF trick modes support in both GStreamer RTSP server and client
- Support for the various trick modes described in section 6 of the
ONVIF streaming spec has been implemented in both gst-rtsp-server
and rtspsrc.
- Various new properties in rtspsrc must be set to take advantage of
the ONVIF support
- Examples are available here: test-onvif-server.c and
test-onvif-client.c
- Watch Mathieu Duponchelle’s talk “Implementing a Trickmode Player
with ONVIF, RTSP and GStreamer” for more information and a live
demo.
GStreamer Codecs library with decoder base classes
This introduces a new library in gst-plugins-bad which contains a set of
base classes that handle bitstream parsing and state tracking for the
purpose of decoding different codecs. Currently H264, H265, VP8 and VP9
are supported. These bases classes are meant primarily for internal use
in GStreamer and are used in various decoder elements in connection with
low level decoding APIs like DXVA, NVDEC, VAAPI and V4L2 State Less
decoders. The new library is named gstreamer-codecs-1.0 /
libgstcodecs-1.0 and is not yet guaranteed to be API stable across major
versions.
MPEG-TS muxing improvements
The GStreamer MPEG-TS muxer has seen major improvements on various
fronts in this cycle:
- It has been ported to the GstAggregator base class which means it
can work in defined-latency mode with live input sources and
continue streaming if one of the inputs stops producing data.
- atscmux, a new ATSC-specific tsmux subclass
- Constant Bit Rate (CBR) muxing support via the new bitrate property
which allows setting the target bitrate in bps. If this is set the
muxer will insert null packets as padding to achieve the desired
multiplex-wide constant bitrate.
- compliance fixes for TV broadcasting use cases (esp. ATSC). See
Jan’s talk “TV Broadcast compliant MPEG-TS” for details.
- Streams can now be added and removed at runtime: Until now, any
streams in tsmux had to be present when the element started
outputting its first buffer. Now they can appear at any point during
the stream, or even disappear and reappear later using the same PID.
- new pcr-interval property allows applications to configure the
desired interval instead of hardcoding it
- basic SCTE-35 support. This is enabled by setting the scte-35-pid
property on the muxer. Sending SCTE-35 commands is then done by
creating the appropriate SCTE-35 GstMpegtsSection and sending them
on the muxer.
- MPEG-2 AAC handling improvements
New elements
- New qmlgloverlay element for rendering a QtQuick scene over the top
of a video stream. qmlgloverlay requires that Qt support adopting an
external OpenGL context and is known to work on X11 and Windows.
Wayland is known not to work due to limitations within Qt. Check out
the example to see how it works.
- The clocksync element is a generic element that can be placed in a
pipeline to synchronise passing buffers to the clock at that point.
This is similar to identity sync=true, but because it isn’t
GstBaseTransform-based, it can process GstBufferLists without
breaking them into separate GstBuffers. It is also more discoverable
than the identity option. Note that you do not need to insert this
element into your pipeline to make GStreamer sync to the pipeline
clock, this is usually handled automatically by the elements in the
pipeline (sources and sinks mostly). This element is useful to feed
non-live input such as local files into elements that expect live
input such as webrtcbin.`
- New imagesequencesrc element to easily create a video stream from a
sequence of JPEG or PNG images (or any other encoding where the type
can be detected), basically a multifilesrc made specifically for
image sequences.
- rpicamsrc element for capturing raw or encoded video (H.264, MJPEG)
from the Raspberry Pi camera. This works much like the popular
raspivid command line utility but outputs data nicely timestamped
and formatted in order to integrate nicely with other GStreamer
elements. Also comes with a device provider so applications can
discover the camera if available.
- aatv and cacatv video filters that transform video ASCII art style
- avtp: new Audio Video Transport Protocol (AVTP) plugin for Linux.
See Andre Guedes’ talk “Audio/Video Bridging (AVB) support in
GStreamer” for more details.
- clockselect: a pipeline element that enables clock selection/forcing
via gst-launch pipeline syntax.
- dashsink: Add new sink to produce DASH content. See Stéphane’s talk
or blog post for details.
- dvbsubenc: a DVB subtitle encoder element
- microdns: a libmicrodns-based mdns device provider to discover RTSP
cameras on the local network
- mlaudiosink: new audio sink element for the Magic Leap platform,
accompanied by an MLSDK implementation in the amc plugin
- msdkvp9enc: VP9 encoder element for the Intel MediaSDK
- rist: new plugin implementing support for the Video Services Forum’s
Reliable Internet Stream Transport (RIST) TR-06-1 Simple Profile.
See Nicolas’ blog post “GStreamer support for the RIST
Specification” for more details.
- rtmp2: new RTMP client source and sink elements with fully
asynchronous network operations, better robustness and additional
features such as handling ping and stats messages, and adobe-style
authentication. The new rtmp2src and rtmp2sink elements should be
API-compatible with the old rtmpsrc / rtmpsink elements and should
work as drop-in replacements.
- new RTP source and sink elements to easily set up RTP streaming via
rtp:// URIs: The rtpsink and rtpsrc elements add an URI interface so
that streams can be decoded with decodebin using rtp:// URIs. These
can be used as follows: ``` gst-launch-1.0 videotestsrc ! x264enc !
rtph264pay config-interval=3 ! rtpsink uri=rtp://239.1.1.1:1234
gst-launch-1.0 videotestsrc ! x264enc ! rtph264pay config-interval=1
! rtpsink uri=rtp://239.1.2.3:5000 gst-launch-1.0 rtpsrc
uri=rtp://239.1.2.3:5000?encoding-name=H264 ! rtph264depay !
avdec_h264 ! videoconvert ! xvimagesink
gst-launch-1.0 videotestsrc ! avenc_mpeg4 ! rtpmp4vpay
config-interval=1 ! rtpsink uri=rtp://239.1.2.3:5000 gst-launch-1.0
rtpsrc uri=rtp://239.1.2.3:5000?encoding-name=MP4V-ES ! rtpmp4vdepay
! avdec_mpeg4 ! videoconvert ! xvimagesink ```
- svthevcenc: new SVT-HEVC-based H.265 video encoder
- switchbin: new helper element which chooses between a set of
processing chains (paths) based on input caps, and changes the
active chain if new caps arrive. Paths are child objects, which are
accessed by the GstChildProxy interface. See the switchbin
documentation for a usage example.
- vah264dec: new experimental va plugin with an element for H.264
decoding with VA-API using GStreamer’s new stateless decoder
infrastructure (see Linux section below).
- v4l2codecs: introduce an V4L2 CODECs Accelerator supporting the new
CODECs uAPI in the Linux kernel (see Linux section below)
- zxing new plugin to detect QR codes and barcodes, based on libzxing
- also see the Rust plugins section below which contains plenty of new
exciting plugins written in Rust!
New element features and additions
GStreamer core
- filesink: Add a new “full” buffer mode. Previously the default and
full modes were the same. Now the default mode is like before: it
accumulates all buffers in a buffer list until the threshold is
reached and then writes them all out, potentially in multiple
writes. The new full mode works by always copying memory to a single
memory area and writing everything out with a single write once the
threshold is reached.
- multiqueue: Add stats property and
current-level-{buffers, bytes, time} pad properties to query the
current levels of the corresponding internal queue.
Plugins Base
- alsa: implement a device provider
- alsasrc: added use-driver-timestamp property to force use of
pipeline timestamps (and disable driver timestamps) if so desired
- audioconvert: fix changing the mix-matrix property at runtime
- appsrc: added support for segment forwarding or custom GstSegments
via GstSample, enabled via the handle-segment-change property. This
only works for segments in TIME format for now.
- compositor: various performance optimisations, checkerboard drawing
fixes, and support for VUYA format
- encodebin: Fix and refactor smart encoding; ensure that a single
segment is pushed into encoders; improve force-key-unit event
handling.
- opusenc: Add low delay option (audio-type=restricted-lowdelay) to
disable the SILK layer and achieve only 5ms delay.
- opusdec: add stats property to retrieve various decoder statistics.
- uridecodebin3: Let decodebin3 do its stream selection if no one
answers
- decodebin3: Avoid overriding explicit user selection of streams
- playbin: add flag to force use of software decoders over any
hardware decoders that might also be available
- playbin3, playbin: propagate sink context
- rawvideoparse: Fix tiling support, allow setting colorimetry
- subparse: output plain utf8 text instead of pango-markup formatted
text if downstream requires it, useful for interop with elements
that only accept utf8-formatted subtitles such as muxers or closed
caption converters.
- tcpserversrc, tcpclientsrc: add stats property with TCP connection
stats (some are only available on Linux though)
- timeoverlay: add show-times-as-dates, datetime-format and
datetime-epoch properties to display times with dates
- videorate: Fix changing rate property during playback; reverse
playback fixes; update QoS events taking into account our rate
- videoscale: pass through and transform size sensitive metas instead
of just dropping them
Plugins Good
- avidemux can handle H.265 video now. Our advice remains to
immediately cease all contact and communication with anyone who
hands you H.265 video in an AVI container, however.
- avimux: Add support for S24LE and S32LE raw audio and v210 raw video
formats; support more than 2 channels of raw audio.
- souphttpsrc: disable session sharing and cookie jar when the cookies
property is set; correctly handle seeks past the end of the content
- deinterlace: new YADIF deinterlace method which should provide
better quality than the existing methods and is LGPL licensed;
alternate fields are supported as input to the deinterlacer as well
now, and there were also fixes for switching the deinterlace mode on
the fly.
- flvmux: in streamable mode allow adding new pads even if the initial
header has already been written. Old clients will only process the
initial stream, new clients will get a header with the new streams.
The skip-backwards-streams property can be used to force flvmux to
skip and drop a few buffers rather than produce timestamps that go
backward and confuse librtmp-based clients. There’s also better
handling for timestamp rollover when streaming for a long time.
- imagefreeze: Add live mode, which can be enabled via the new is-live
property. In this mode frames will only be output in PLAYING state
according to the negotiated framerate, skipping frames if the output
can’t keep up (e.g. because it’s blocked downstream). This makes it
possible to actually use imagefreeze in live pipelines without
having to manually ensure somehow that it starts outputting at the
current running time and without still risking to fall behind
without recovery.
- matroskademux, qtdemux: Provide audio lead-in for some lossy formats
when doing accurate seeks, to make sure we can actually decode
samples at the desired position. This is especially important for
non-linear audio/video editing use-cases.
- matroskademux, matroskamux: Handle interlaced field order (tff, bff)
- matroskamux:
- new offset-to-zero property to offset all streams to start at
zero. This takes the timestamp of the earliest stream and
offsets it so that it starts at 0. Some software (VLC,
ffmpeg-based) does not properly handle Matroska files that start
at timestamps much bigger than zero, which could happen with
live streams.
- added a creation-time property to explicitly set the creation
time to write into the file headers. Useful when remuxing, for
example, but also for live feeds where the DateUTC header can be
set a UTC timestamp corresponding to the beginning of the file.
- the muxer now also always waits for caps on sparse streams, and
warns if caps arrive after the header has already been sent,
otherwise the subtitle track might be silently absent in the
final file. This might affect applications that send sparse data
into matroskamux via an appsrc element, which will usually not
send out the initial caps before it sends out the first buffer.
- pulseaudio: device provider improvements: fix discovery of
newly-added devices and hide the alsa device provider if we provide
alsa devices
- qtdemux: raw audio handling improvements, support for AC4 audio, and
key-units trickmode interval support
- qtmux:
- was ported to the GstAggregator base class which allows for
better handling of live inputs, but might entail minor
behavioural changes for sparse inputs if inputs are not live.
- has also gained a force-create-timecode-trak property to create
a timecode trak in non-mov flavors, which may not be supported
by Apple but is supported by other software such as Final Cut
Pro X
- also a force-chunks property to force the creation of chunks
even in single-stream files, which is required for Apple ProRes
certification.
- also supports 8k resolutions in prefill mode with ProRes.
- rtpbin gained a request-jitterbuffer signal which allows
applications to plug in their own jitterbuffer implementation such
as the threadsharing jitterbuffer from the Rust plugins, for
example.
- rtprtxsend: add clock-rate-map property to allow generic RTP input
caps without a clock-rate whilst still supporting the max-size-time
property for bundled streams.
- rtpssrcdemux: introduce max-streams property to guard against
attacks where the sender changes SSRC for every RTP packet.
- rtph264pay, rtph264pay: implement STAP-A and various aggregation
modes controled by the new aggegrate-mode property: none to not
aggregate NAL units (as before), zero-latency to aggregate NAL units
until a VCL or suffix unit is included, or max to aggregate all NAL
units with the same timestamp (which adds one frame of latency). The
default has been kept at none for backwards compatibility reasons
and because various RTP/RTSP implementions don’t handle aggregation
well. For WebRTC use cases this should be set to zero-latency,
however.
- rtpmp4vpay: add support for config-interval=-1 to resend headers
with each IDR keyframe, like other video payloaders.
- rtpvp8depay: Add wait-for-keyframe property for waiting until the
next keyframe after packet loss. Useful if the video stream was not
encoded with error resilience enabled, in which case packet loss
tends to cause very bad artefacts when decoding, and waiting for the
next keyframe instead improves user experience considerably.
- splitmuxsink and splitmuxsrc can now handle auxiliary video streams
in addition to the primary video stream. The primary video stream is
still used to select fragment cut points at keyframe boundaries.
Auxilliary video streams may be broken up at any packet - so
fragments may not start with a keyframe for those streams.
- splitmuxsink:
- new muxer-preset and sink-preset properties for setting
muxer/sink presets
- a new start-index property to set the initial fragment id
- and a new muxer-pad-map property which explicitly maps
splitmuxsink pads to the muxer pads they should connect to,
overriding the implicit logic that tries to match pads but
yields arbitrary names.
- Also includes the actual sink element in the fragment-opened and
fragment-closed element messages now, which is especially useful
for sinks without a location property or when finalisation of
the fragments is done asynchronously.
- videocrop: add support for Y444, Y41B and Y42B pixel formats
- vp8enc, vp9enc: change default value of VP8E_SET_STATIC_THRESHOLD
from 0 to 1 which matches what Google WebRTC does and results in
lower CPU usage; also added a new bit-per-pixel property to select a
better default bitrate
- v4l2: add support for ABGR, xBGR, RGBA, and RGBx formats and for
handling interlaced video in alternate fields interlace mode (one
field per buffer instead of one frame per picture with both fields
interleaved)
- v4l2: Profile and level probing support for H264, H265, MPEG-4,
MPEG-2, VP8, and VP9 video encoders and decoders
Plugins Ugly
- asfdemux: extract more metadata: disc number and disc count
- x264enc:
- respect YouTube bitrate recommendation when user sets the
YouTube profile preset
- separate high-10 video formats from 8-bit formats to improve
depth negotiation and only advertise suitable input raw formats
for the desired output depth
- forward downstream colorimetry and chroma-site restrictions to
upstream elements
- support more color primaries/mappings
Plugins Bad
- av1enc: add threads, row-mt and tile-{columns,rows} properties for
this AOMedia AV1 encoder
- ccconverter: implement support for CDP framerate conversions
- ccextractor: Add remove-caption-meta property to remove caption
metas from the outgoing video buffers
- decklink: add support for 2K DCI video modes, widescreen NTSC/PAL,
and for parsing/outputting AFD/Bar data. Also implement a simple
device provider for Decklink devices.
- dtlsrtpenc: add rtp-sync property which synchronises RTP streams to
the pipeline clock before passing them to funnel for merging with
RTCP.
- fdkaac: also decode MPEG-2 AAC; encoder now supports more
multichannel/surround sound layouts
- hlssink2: add action signals for custom playlist/fragment handling:
Instead of always going through the file system API we allow the
application to modify the behaviour. For the playlist itself and
fragments, the application can provide a GOutputStream. In addition
the sink notifies the application whenever a fragment can be
deleted.
- interlace: can now output data in alternate fields mode; added field
switching mode for 2:2 field pattern
- iqa: Add a mode property to enable strict mode that checks that all
the input streams have the exact same number of frames; also
implement the child proxy interface
- mpeg2enc: add disable-encode-retries property for lower CPU usage
- mpeg4videoparse: allow re-sending codec config at IDR via
config-interval=-1
- mpegtsparse: new alignment property to determine number of TS
packets per output buffer, useful for feeding an MPEG-TS stream for
sending via udpsink. This can be used in combination with the
split-on-rai property that makes sure to start a new output buffer
for any TS packet with the Random Access Indicator set. Also set
delta unit buffer flag on non-random-access buffers.
- mpegdemux: add an ignore-scr property to ignore the SCR in
non-compliant MPEG-PS streams with a broken SCR, which will work as
long as PTS/DTS in the PES header is consistently increasing.
- tsdemux:
- add an ignore-pcr property to ignore MPEG-TS streams with broken
PCR streams on which we can’t reliably recover correct
timestamps.
- new latency property to allow applications to lower the
advertised worst-case latency of 700ms if they know their
streams support this (must have timestamps in higher frequency
than required by the spec)
- support for AC4 audio
- msdk - Intel Media SDK plugin for hardware-accelerated video
decoding and encoding on Windows and Linux:
- mappings for more video formats: Y210, Y410, P012_LE, Y212_LE
- encoders now support bitrate changes and input format changes in
playing state
- msdkh264enc, msdkh265enc: add support for CEA708 closed caption
insertion
- msdkh264enc, msdkh265enc: set Region of Interest (ROI) region
from ROI metas
- msdkh264enc, msdkh265enc: new tune property to enable low-power
mode
- msdkh265enc: add support 12-bit 4:2:0 encoding and 8-bit 4:2:2
encoding and VUYA, Y210, and Y410 as input formats
- msdkh265enc: add support for screen content coding extension
- msdkh265dec: add support for main-12/main-12-intra,
main-422-10/main-422-10-intra 10bit,
main-422-10/main-422-10-intra 8bit,
main-422-12/main-422-12-intra, main-444-10/main-444-10-intra,
main-444-12/main-444-12-intra, and main-444 profiles
- msdkvp9dec: add support for 12-bit 4:4:4
- msdkvpp: add support for Y410 and Y210 formats, cropping via
properties, and a new video-direction property.
- mxf: Add support for CEA-708 CDP from S436 essence tracks. mxfdemux
can now handle Apple ProRes
- nvdec: add H264 + H265 stateless codec implementation nvh264sldec
and nvh265sldec with fewer features but improved latency. You can
set the environment variable GST_USE_NV_STATELESS_CODEC=h264 to use
the stateless decoder variant as nvh264dec instead of the “normal”
NVDEC decoder implementation.
- nvdec: add support for 12-bit 4:4:4/4:2:0 and 10-bit 4:2:0 decoding
- nvenc:
- add more rate-control options, support for B-frame encoding (if
device supports it), an aud property to toggle Access Unit
Delimiter insertion, and qp-{min,max,const}-{i,p,b} properties.
- the weighted-pred property enables weighted prediction.
- support for more input formats, namely 8-bit and 10-bit RGB
formats (BGRA, RGBA, RGB10A2, BGR10A2) and YV12 and VUYA.
- on-the-fly resolution changes are now supported as well.
- in case there are multiple GPUs on the system, there are also
per-GPU elements registered now, since different devices will
have different capabilities.
- nvh265enc can now support 10-bit YUV 4:4:4 encoding and 8-bit
4:4:4 / 10-bit 4:2:0 formats up to 8K resolution (with some
devices). In case of HDR content HDR related SEI nals will be
inserted automatically.
- openjpeg: enable multi-threaded decoding and add support for
sub-frame encoding (for lower latency)
- rtponviftimestamp: add opt-out “drop-out-of-segment” property
- spanplc: new stats property
- srt: add support for IPv6 and for using hostnames instead of IP
addresses; add streamid property, but also allow passing the id via
the stream URI; add wait-for-connection property to srtsink
- timecodestamper: this element was rewritten with an updated API
(properties); it has gained many new properties, seeking support and
support for linear timecode (LTC) from an audio stream.
- uvch264src now comes with a device provider to advertise available
camera sources that support this interface (mostly Logitech C920s)
- wpe: Add software rendering support and support for mouse scroll
events
- x265enc: support more 8/10/12 bits 4:2:0, 4:2:2 and 4:4:4 profiles;
add support for mastering display info and content light level
encoding SEIs
gst-libav
- Add mapping for SpeedHQ video codec used by NDI
- Add mapping for aptX and aptX-HD
- avivf_mux: support VP9 and AV1
- avvidenc: shift output buffer timestamps and output segment by 1h
just like x264enc does, to allow for negative DTS.
- avviddec: Limit default number of decoder threads on systems with
more than 16 cores, as the number of threads used in avdec has a
direct impact on the latency of the decoder, which is of as many
frames as threads, so a large numbers of threads can make for
latency levels that can be problematic in some applications.
- avviddec: Add thread-type property that allows applications to
specify the preferred multithreading method (auto, frame, slice).
Note that thread-type=frame may introduce additional latency
especially in live pipelines, since it introduces a decoding delay
of number of thread frames.
Plugin and library moves
- There were no plugin moves or library moves in this cycle.
- The rpicamsrc element was moved into -good from an external
repository on github.
Plugin removals
The following elements or plugins have been removed:
- The yadif video deinterlacing plugin from gst-plugins-bad, which was
one of the few GPL licensed plugins, has been removed in favour of
deinterlace method=yadif.
- The avdec_cdgraphics CD Graphics video decoder element from
gst-libav was never usable in GStreamer and we now have a cdgdec
element written in Rust in gst-plugins-rs to replace it.
- The VDPAU plugin has been unmaintained and unsupported for a very
long time and does not have the feature set we expect from
hardware-accelerated video decoders. It’s been superseded by the
nvcodec plugin leveraging NVIDIA’s NVDEC API.
Miscellaneous API additions
GStreamer core
- gst_task_resume(): This new API allows resuming a task if it was
paused, while leaving it in stopped state if it was stopped or not
started yet. This can be useful for callback-based driver workflows,
where you basically want to pause and resume the task when buffers
are notified while avoiding the race with a gst_task_stop() coming
from another thread.
- info: add printf extensions GST_TIMEP_FORMAT and GST_STIMEP_FORMAT
for printing GstClockTime/GstClockTimeDiff pointers, which is much
more convenient to use in debug log statements than the usual
GST_TIME_FORMAT-followed-by-GST_TIME_ARGS dance. Also add an
explicit GST_STACK_TRACE_SHOW_NONE enum value.
- gst_element_get_current_clock_time() and
gst_element_get_current_running_time(): new helper functions for
getting an element clock’s time, and the clock time minus base time,
respectively. Useful when adding additional input branches to
elements such as compositor, audiomixer, flvmux, interleave or
input-selector to determine initial pad offsets and such.
- seeking: Add GST_SEEK_FLAG_TRICKMODE_FORWARD_PREDICTED to just skip
B-frames during trick mode, showing both keyframes + P-frame, and
add support for it in h264parse and h265parse.
- elementfactory: add GST_ELEMENT_FACTORY_TYPE_HARDWARE to allow
elements to advertise that they are hardware-based or interact with
hardware. This has multiple applications:
- it makes it possible to easily differentiate hardware and
software based element implementations such as audio or video
encoders and decoders. This is useful in order to force the use
of software decoders for specific use cases, or to check if a
selected decoder is actually hardware-accelerated or not.
- elements interacting with hardware and their respective drivers
typically don’t know the actually supported capabilities until
the element is set into at least READY state and can open a
device handle and probe the hardware.
- gst_uri_from_string_escaped(): identical to gst_uri_from_string()
except that the userinfo and fragment components of the URI will not
be unescaped while parsing. This is needed for correctly parsing
usernames or passwords with : in them .
- paramspecs: new GstParamSpec flag GST_PARAM_CONDITIONALLY_AVAILABLE
to indicate that a property might not always exist.
- gst_bin_iterate_all_by_element_factory_name() finds elements in a
bin by factory name
- pad: gst_pad_get_single_internal_link() is a new convenience
function to return the single internal link of a pad, which is
useful e.g. to retrieve the output pad of a new multiqueue request
pad.
- datetime: Add constructors to create datetimes with timestamps in
microseconds, gst_date_time_new_from_unix_epoch_local_time_usecs()
and gst_date_time_new_from_unix_epoch_utc_usecs().
- gst_debug_log_get_lines() gets debug log lines formatted in the same
way the default log handler would print them
- GstSystemClock: Add GST_CLOCK_TYPE_TAI as GStreamer abstraction for
CLOCK_TAI, to support transmission offloading features where network
packets are timestamped with the time they are deemed to be actually
transmitted. Useful in combination with the new AVTP plugin.
- miscellaneous utility functions: gst_clear_uri(),
gst_structure_take().
- harness: Added gst_harness_pull_until_eos()
- GstBaseSrc:
- gst_base_src_new_segment() allows subclasses to update the
segment to be used at runtime from the ::create() function. This
deprecates gst_base_src_new_seamless_segment()
- gst_base_src_negotiate() allows subclasses to trigger format
renegotiation at runtime from inside the ::create() or ::alloc()
function
- GstBaseSink: new stats property and gst_base_sink_get_stats() method
to retrieve various statistics such as average frame rate and
dropped/rendered buffers.
- GstBaseTransform: gst_base_transform_reconfigure() is now public
API, useful for subclasses that need to completely re-implement the
::submit_input_buffer() virtual method
- GstAggregator:
- gst_aggregator_update_segment() allows subclasses to update the
output segment at runtime. Subclasses should use this function
rather than push a segment event onto the source pad directly.
- new sample selection API:
- subclasses should now call gst_aggregator_selected_samples()
from their ::aggregate() implementation to signal that they
have selected the next samples they will aggregate
- GstAggregator will then emit the samples-selected signal
where handlers can then look up samples per pad via
gst_aggregator_peek_next_sample().
- This is useful for example to atomically update input pad
properties in mixer subclasses such as compositor.
Applications can now update properties with precise control
of when these changes will take effect, and for which input
buffer(s).
- gst_aggregator_finish_buffer_list() allows subclasses to push
out a buffer list, improving efficiency in some cases.
- a ::negotiate() virtual method was added, for consistency with
other base classes and to allow subclasses to completely
override the negotiation behaviour.
- the new ::sink_event_pre_queue() and ::sink_query_pre_queue()
virtual methods allow subclasses to intercept or handle
serialized events and queries before they’re queued up
internally.
GStreamer Plugins Base Libraries
Audio library
- audioaggregator, audiomixer: new output-buffer-duration-fraction
property which allows use cases such as keeping the buffers output
by compositor on one branch and audiomixer on another perfectly
aligned, by requiring the compositor to output a n/d frame rate, and
setting output-buffer-duration-fraction to d/n on the audiomixer.
- GstAudioDecoder: new max-errors property so applications can
configure at what point the decoder should error out, or tell it to
just keep going
- gst_audio_make_raw_caps() and gst_audio_formats_raw() are
bindings-friendly versions of the GST_AUDIO_CAPS_MAKE() C macro.
- gst_audio_info_from_caps() now handles encoded audio formats as well
PbUtils library
- GstEncodingProfile:
- Do not restrict number of similar profiles in a container
- add GstValue serialization function
- codec utils now support more H.264/H.265 profiles/levels and have
improved extension handling
RTP library
- rtpbasepayloader: Add scale-rtptime property for scaling RTP
timestamp according to the segment rate (equivalent to RTSP speed
parameter). This is useful for ONVIF trickmodes via RTSP.
- rtpbasepayload: add experimental property for embedding twcc
sequencenumbers for Transport-Wide Congestion Control (gated behind
the GST_RTP_ENABLE_EXPERIMENTAL_TWCC_PROPERTY environment
variable) - more generic API for enabling this is expected to land
in the next development cycle.
- rtcpbuffer: add RTPFB_TYPE_TWCC for Transport-Wide Congestion
Control
- rtpbuffer: add
gst_rtp_buffer_get_extension_onebyte_header_from_bytes()``, so that one can parse theGBytes`
returned by gst_rtp_buffer_get_extension_bytes()
- rtpbasedepayload: Add max-reorder property to make the
previously-hardcoded value when to consider a sender to have
restarted configurable. In some scenarios it’s particularly useful
to set max-reorder=0 to disable the behaviour that the depayloader
will drop packets: when max-reorder is set to 0 all
reordered/duplicate packets are considered coming from a restarted
sender.
RTSP library
- add gst_rtsp_url_get_request_uri_with_control() to create request
uri combined with control url
- GstRTSPConnection: add the possibility to limit the Content-Length
for RTSP messages via
gst_rtsp_connection_set_content_length_limit(). The same
functionality is also exposed in gst-rtsp-server.
SDP library
- add support for parsing the extmap attribute from caps and storing
inside caps The extmap attribute allows mapping RTP extension header
IDs to well-known RTP extension header specifications. See RFC8285
for details.
Tags library
- update to latest iso-code and support more languages
- add tags for acoustid id & acoustid fingerprint, plus MusicBrainz ID
handling fixes
Video library
- High Dynamic Range (HDR) video information representation and
signalling enhancements:
- New APIs for HDR video information representation and
signalling:
- GstVideoMasteringDisplayInfo: display color volume info as
per SMPTE ST 2086
- GstVideoContentLightLevel: content light level specified in
CEA-861.3, Appendix A.
- plus functions to serialise/deserialise and add them to or
parse them from caps
- gst_video_color_{matrix,primaries,transfer}_{to,from}_iso():
new utilility functions for conversion from/to ISO/IEC
23001-8
- add ARIB STD-B67 transfer chracteristic function
- add SMPTE ST 2084 support and BT 2100 colorimetry
- define bt2020-10 transfer characteristics for clarity:
bt707, bt2020-10, and bt2020-12 transfer characteristics are
functionally identical but have their own unique values in
the specification.
- h264parse, h265parse: Parse mastering display info and content
light level from SEIs.
- matroskademux: parse HDR metadata
- matroskamux: Write MasteringMetadata and Max{CLL,FALL}. Enable
muxing with HDR meta data if upstream provided it
- avviddec: Extract HDR information if any and map bt2020-10, PQ
and HLG transfer functions
- added bt601 transfer function (for completeness)
- support for more pixel formats:
- Y412 (packed 12 bits 4:4:4:4)
- Y212 (packed 12 bits 4:2:2)
- P012 (semi-planar 4:2:0)
- P016_{LE,BE} (semi-planar 16 bits 4:2:0)
- Y444_16{LE,BE} (planar 16 bits 4:4:4)
- RGB10A2_LE (packed 10-bit RGB with 2-bit alpha channel)
- NV12_32L32 (NV12 with 32x32 tiles in linear order)
- NV12_4L4 (NV12 with 4x4 tiles in linear order)
- GstVideoDecoder:
- new max-errors property so applications can configure at what
point the decoder should error out, or tell it to just keep
going
- new qos property to disable dropping frames because of QoS, and
post QoS messages on the bus when dropping frames. This is
useful for example in a scenario where the decoded video is
tee-ed off to go into a live sink that syncs to the clock in one