Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for VP9 and H.264 profile negotiation #2080

Merged
merged 8 commits into from
May 18, 2020
Merged

Conversation

lminiero
Copy link
Member

@lminiero lminiero commented Apr 17, 2020

As the title says, this adds support for profile negotiation to Janus, namely for VP9 and H.264. In fact, if you ever looked at SDPs, you might have noticed that the same codec may have different payload types associated to them: when this happens, it's because there's a negotiation of different "flavours" of the codec, mostly via profiles.

To make simple example, here is what Chrome might offer for VP9:

	a=rtpmap:98 VP9/90000
	a=rtcp-fb:98 goog-remb
	a=rtcp-fb:98 transport-cc
	a=rtcp-fb:98 ccm fir
	a=rtcp-fb:98 nack
	a=rtcp-fb:98 nack pli
	a=fmtp:98 profile-id=0
	a=rtpmap:99 rtx/90000
	a=fmtp:99 apt=98
	a=rtpmap:100 VP9/90000
	a=rtcp-fb:100 goog-remb
	a=rtcp-fb:100 transport-cc
	a=rtcp-fb:100 ccm fir
	a=rtcp-fb:100 nack
	a=rtcp-fb:100 nack pli
	a=fmtp:100 profile-id=2
	a=rtpmap:101 rtx/90000
	a=fmtp:101 apt=100

As you can see, VP9 is there twice, but one if profile 0 (payload type 98 and rtx 99), and the other is profile 2 (payload type 100 and rtx 101), and they're not the same thing. H.264 has even more options, usually. So far, in Janus we've basically ignored any codec duplicate: we'd just stick to the first payoad type we met. This patch tries to address that, allowing plugins to express preferences when it comes to offers and answers they generate.

An easy way to test this in the demos is using the vcodec and vprofile query strings in the EchoTest demo: if you open echotest.html?vcodec=vp9 you'll just negotiate VP9 as usual (no preference), which means we'll pick first payload type we meet; opening echotest.html?vcodec=vp9&vprofile=2, instead, will tell the plugin to prefer profile=2, and so the session will end up using the other one. You can play the same way with H.264, e.g., via echotest.html?vcodec=h264 vs. echotest.html?vcodec=h264&vprofile=42001f vs. echotest.html?vcodec=h264&vprofile=42e01f.

VideoRoom, Record&Play, Lua and Duktape plugins have all both been updated to support this feature: other plugins don't need it, as negotiation isn't up to them (e.g., VideoCall, where browsers choose what to use). The VideoRoom in particular now provides ways to configure the desired profiles when creating a room, pretty much as you can choose the codecs you want to be used.

One thing worth mentioning is that this patch doesn't save the profile info in .mjr files yet, which it probably should: while profile negotiation has a relative impact on H.264, the VP9 profile in use actually changes the bitstream considerably, and so trying to replay a VP9/profile=2 recording via Record&Play using profile=0 in the SDP will result in the video not to work at all. I plan to work on this later on, as it would require changes to the other plugins as well (e.g., if the VideoCall decided to use profile=2 and record the call, then we should know).

Feedback welcome!

@lminiero lminiero changed the title Add support for Vp9 and H.264 profile negotiation Add support for VP9 and H.264 profile negotiation Apr 17, 2020
@lminiero
Copy link
Member Author

One thing worth mentioning is that this patch doesn't save the profile info in .mjr files yet, which it probably should: while profile negotiation has a relative impact on H.264, the VP9 profile in use actually changes the bitstream considerably, and so trying to replay a VP9/profile=2 recording via Record&Play using profile=0 in the SDP will result in the video not to work at all. I plan to work on this later on, as it would require changes to the other plugins as well (e.g., if the VideoCall decided to use profile=2 and record the call, then we should know).

This has now been done for the main plugins, so I consider this PR complete. Pending some tests and feedback, I plan to merge soon.

@lminiero
Copy link
Member Author

Merging.

@lminiero lminiero merged commit fa12368 into master May 18, 2020
@lminiero lminiero deleted the codec-profiles branch May 18, 2020 08:34
@oscarvadillog
Copy link
Contributor

oscarvadillog commented May 20, 2020

Can this fix a problem I had when I enable video in the SIP plugin? With Firefox or Chrome mobile I am not able to watch the remote video. With Chrome it was fine.

Example SIP URI: sip:president@selfie.vc

It seems it works now with Firefox and the last Janus demos

@lminiero
Copy link
Member Author

Only one way to know... 😉

@oscarvadillog
Copy link
Contributor

Yep, now it is working properly with the master branch. Just only to understand where was fixed.
Thanks Lorenzo! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants