Video transition example not working in Chrome for Android

[georgwaechter]

Hi,

i've noticed that the video transition example does not work on Chrome for Android. It blends the video as expected, but the charachter is not moving. It looks like two images - instead of videos - get transitioned.

I've used the Chrome Developer tools to inspect the error log and came to this message:
videocontext.js:1623 Uncaught (in promise) DOMException: play() can only be initiated by a user gesture

The problem is that the play() cal is not within the stack trace of the click() handler for the play button. Thus, Chrome does not allow it to play.

I've read some comments in the chrome issue tracker and there is a workaround that seems to work for tags. It may work also for video (haven't had time to test it yet): Create an empty

By the way: On Firefox for Android it works as expected. But i've read that mobile safari has the same restrictions as Chrome for Android. Is here somebody with an iPhone device to test this?

Georg

[georgwaechter]

Screenshot of developer console output

[emshotton]

Hi Georg,

Thank you for raising this! That's a promising solution to the issue, it should be fairly easy to try, as it's possible to create a VideoNode by passing in an existing element to the createVideoNode call. i.e:

var videoContext = new VideoContext();
var videoElement = document.createElement("video");
var videoNode = videoContext.createVideoSourceNode(videoElement);

You should also be able to use callbacks on the video node to set the src when the videoNode is due for loading:

videoNode.registerCallback("load", function(){
    videoNode.element.src = "./video.mp4";
})

Given play has already been called on the node, i'm not sure how setting the src attribute of the underlying element effects behaviour. We also wouldn't want the node to start playing straight away as it will have been sequenced to start playing a defined period after the load callback gets called (defaults to 4 seconds). If the loading of the video is fast enough to not cause perceptible lag a callback could be attached to the "play" callback for the node instead of the "load" callback.

Kind regards,
Matt

[emshotton]

Also, it's good to know it works on Firefox on android! We've tried it on an IPad previously with no success, not tried an IPhone yet.

[georgwaechter]

Hi Matt,

i've tested the behavior more in depth, this is what I learned: It is not sufficient to create the

Conclusion: We dont need the "load" callback. This should be enough:

var videoContext = new VideoContext();
var videoElement = document.createElement("video");
videoElement.play(); // does nothing as we dont have a src
var videoNode = videoContext.createVideoSourceNode(videoElement, ..);

As an alternative to play() you can also create an autoplay tag: <video autoplay... which has the same effect as calling play() manually.

Is there a repository for the examples? I would create a pull request to enable this apprach ..

[emshotton]

Hi Georg,

Fantastic, that's great to know. There is a seperate gh-pages branch which hosts the examples:

https://github.com/bbc/VideoContext/tree/gh-pages

Feel free to make a pull request to that and I'll merge it in.

One thought about this approach; there's a slight limitation in that it will require knowing the number of videos you plan on playing up-front or managing a "pool" of activated video elements as the VideoContext can potentially sequence new video sources at any time. That said, this is far better than not having any mobile support at all.

[georgwaechter]

Explanation related to chromes “user gesture“: WICG/interventions#12

[georgwaechter]

Matt, is it possible to defer the "load" callback, somehow? I tried to implement the desired behavior, but the "load" event is called durching the construction in the "onload" function.

I either have to move all code into the play button handler (so we loose the visualization on load) or i find a possibility to defer the "load" event. I was not able to find the place where "load" is called within the code ..

[georgwaechter]

Ah i see: The VideoContext ist even running before we call play() ... thus the requestAnimationFrame triggers the "load" event. Hm, can we also create an inactive/disabled VideoContext?

[georgwaechter]

More information about the situation in iOS: https://webkit.org/blog/6784/new-video-policies-for-ios/

[georgwaechter]

Ok, the issue is the following: I can't stop the VideoContext from calling play() right after the construction. This is due to the registerUpdatable call, which starts the render "loop" without any possibility to stop it.

We could move the VideoContext construction into a user gesture event handler like "click", but i think you agree that this is not a nice solution.

I propose to change this behavior, because it is a bit surprising. As a library user I would expect to be the one who is controlling the render loop.

I would like to see a possilibity to either do the registerUpdatable() call manually or to undo it explicitely (a more backward compatible way).

@MatthewShotton What do you like more? I would suggest to export the update() related functions.

[emshotton]

Hi Georg,

I've made some changes to allow passing of options in to the VideoContext's constructor. One of those is "manualUpdate". Setting this to true will prevent the registerUpdatable call from happening and allow the user to handle the update loop of the VideoContext.
To facilitate this "update(dt)" is now exposed by the VideoContext. Where dt is the time in seconds between the current call to update and the previous one.

This example should give an idea of how these changes can be used:

var canvasElement = document.getElemenyById("canvas");
var ctx = new VideoContext(canvasElement, undefined, {"manualUpdate" : true});

var previousTime;
function update(time){
    if (previousTime === undefined) previousTime = time;
    var dt = (time - previousTime)/1000;
    ctx.update(dt);
    previousTime = time;
    requestAnimationFrame(update);
}
update();

For the time being I'm keep the default behaviour as the VideoContext handling the update loop for you as a number of projects rely on this functionality. This can be reviewed when we get to the v1.0.0 release.

Kind regards,
Matt

[georgwaechter]

Hi Matt,

thanks, this is a good solution! I'll try it and post my results here in a few days ..

Best,

Georg

[emshotton]

Initial experiments into adding a cache of reusable video elements which are all initialised on the initial play() call has been added in the mobile-support branch. When enabled new video nodes are created from video elements stored in the cache. Once the nodes have finished playing the video elements can be reused by subsequently created video nodes.

Two new values have also been added to the options passed into the VideoContext constructor. useVideoElementCache and videoElementCacheSize.

let vc = new VideoContext(canvas, undefined, {useVideoElementCache:true, videoElementCacheSize:5});

The current caveat of this approach is it only works if you are dynamically creating video nodes. If you make more video nodes upfront than you have elements in the cache, the cache will get depleted even if those video nodes are not playing at the same time. This will be addressed in a future commit and before the v1.0 release.

(this also affects #16 )