FLV support #10756

kmeisthax · 2023-04-21T02:52:21Z

This PR adds FLV support. Notably, only video: the existing streaming audio APIs want to work with either SWF bitstreams or single contiguous blocks of MP3 data. That needs refactoring big enough to justify a separate PR.

FLV also, bizarrely, has functionality for calling ActionScript methods from the video stream. Yes, the onMetaData event is actually triggered by one of these script calls; and no I did not implement it yet. I still do need to parse this block in order to not violate the video backend's existing API prerequisites, though.

~~This can't be merged until ruffle-rs/h263-rs#50 is merged and this PR is updated to target the merged version.~~ I rolled that PR into this one so it can be merged in one go.

TheGhostboi64 · 2023-04-21T12:17:39Z

this is very cool! A defunct website I recreate mostly uses FLVs for the intros and others, so I can't wait to see it working! Nice!

danielhjacobs · 2023-04-21T13:33:43Z

Progresses http://labcenter.dnalc.org/labs/dnaextraction/dnaextraction_d.html (uses VP6A and MP3 codecs with external FLV files), which used to say:

ERROR core/src/tag_utils.rs:414 Error running definition tag: DefineVideoStream, got Couldn't read SWF
ERROR core/src/library.rs:200 Tried to instantiate non-registered character ID 439
ERROR core/src/display_object/movie_clip.rs:1611 Unable to instantiate display node id 439, reason being: Character id doesn't exist

It now says:

ERROR core/src/streams.rs:398 Decoding video frame [0-4] failed: Decoder error: InvalidData
WARN core/src/display_object/video.rs:565 Video has no decoded frame to render.

There is no functional difference between what used to happen on that site and what happens now though, unfortunately.

Progresses #10173 (uses H.264 codec with external MP4 files), number 2, which used to say:

ERROR core/src/avm2/events.rs:419 Error dispatching event EventObject(EventObject { type: "click", class: flash.events::MouseEvent, ptr: 0x1586748 }) to handler FunctionObject(FunctionObject { ptr: 0x1497430 }) : TypeError: Error #1009: Cannot access a property or method of a null object reference. (accessing field: isWaiting)

It now says:

WARN core/src/stub.rs:126 Encountered stub: AVM2 flash.net.NetConnection.connect()
WARN core/src/stub.rs:126 Encountered stub: AVM2 flash.net.NetStream.client setter
WARN core/src/stub.rs:126 Encountered stub: AVM2 flash.net.NetStream.bufferTime setter
WARN core/src/stub.rs:126 Encountered stub: AVM2 flash.net.NetStream.soundTransform setter
WARN core/src/stub.rs:126 Encountered stub: AVM2 flash.net.NetStream.soundTransform getter
WARN core/src/stub.rs:126 Encountered stub: AVM2 flash.net.NetStream.time getter
WARN core/src/display_object/video.rs:565 Video has no decoded frame to render.

There is no functional difference between what used to happen on that site and what happens now though, unfortunately.

Progresses #10175, number 2, which used to say:

ERROR core/src/display_object/movie_clip.rs:2135 Got "RustError("ArgumentError: Child not a valid display object")" when constructing AVM2 side of movie clip of type fl.video::FLVPlayback
ERROR core/src/display_object/movie_clip.rs:2135 Got "TypeError: Error #1010: A term is undefined and has no properties. (accessing field: TEXT_LAYOUT_FORMAT)

It now says:

core/src/display_object/movie_clip.rs:2135 Got "TypeError: Error #1010: A term is undefined and has no properties. (accessing field: TEXT_LAYOUT_FORMAT)

There is no functional difference between what used to happen on that site and what happens now though, unfortunately.

Progresses #10175, number 3 in the same way described for #10173, number 2.

Lord-McSweeney · 2023-04-21T17:25:20Z

Fixes #10705.

waspennator · 2023-04-21T18:09:45Z

Should progress the flv issue for #10149?

Lord-McSweeney · 2023-04-21T22:02:28Z

I think the reason there's no functional difference currently is because the FLV PR isn't yet merged into the h263-rs repo.

danielhjacobs · 2023-04-21T22:10:51Z

But wasn't the cargo lock changed to point to kmeisthax's latest commit?

kmeisthax · 2023-04-21T22:42:27Z

Daniel is correct, flv-rs is being pulled in by commit ID separate from the other h263-rs packages. Hence why this is still in draft. (or it would be if I hadn't accidentally clicked the regular save button and had a clue as to how to turn on draft mode for a non-draft PR)

Lord-McSweeney · 2023-04-22T03:54:21Z

You can turn it into a real draft by clicking the "Still in progress? Convert to draft" text below Reviewers - Suggestions, which is on the top-right of the page in Github.

kmeisthax · 2023-04-22T18:00:50Z

I love how that link is buried where you can't find it

torokati44 · 2023-04-22T18:05:19Z

FWIW you can also switch the "Open PR" button to open as a draft right away.

core/src/avm2/globals/flash/media/video.rs

iwannabethedev · 2023-05-12T17:15:15Z

core/src/avm2/object/netstream_object.rs

-    .into())
+    .into();
+
+    ns.set_avm_object(activation.context.gc_context, this.into());


Nit: I do not know if I like the cyclic reference between the Object and NetStream, but it might be fine, and alternatives, like a callback, might be overall worse.

iwannabethedev · 2023-05-12T17:17:07Z

core/src/display_object/video.rs

+    movie: Arc<SwfMovie>,
+
+    /// The self bounds for this movie.
+    size: (i32, i32),


~~Nit: Might this be better as (u32, 32)? Might be better as it currently is.~~

core/src/streams.rs

iwannabethedev · 2023-05-12T17:46:21Z

core/src/streams.rs

+                    FlvTagData::Audio(FlvAudioData { .. }) => {
+                        tracing::warn!("Stub: Stream audio processing");
+                    }
+                    FlvTagData::Video(FlvVideoData { codec_id, data, .. }) => {


The function is somewhat long. Would it be helpful (or worse) to maybe put parts of this function into their own functions that is called from here? Like, a function that might be called handle_flv_tag_data_video() for this match?

But, one problem with that is how it would interfere with the loop, in case there are return or break or continue statements and the like. The usage of a loop, together with a lot of code, feels a bit difficult to handle to me, though I could be wrong, both on this point and on possible alternatives on organizing the code being sufficiently better to change to one of them.

iwannabethedev · 2023-05-12T17:54:29Z

It is obvious that a lot of time and effort went into this PR. I know too little of Rust, Ruffle and video playing to really give good feedback on this, but I have tried to leave a few comments. The PR is also rather long, making it more difficult to review. Parts that I think would be good to review is the overall type design and interface between the different parts of the codebase, and maybe also a light review of parts of the implementation, though this is of course only an initial PR and many parts have not been implemented yet or are works in progress.

@kmeisthax What do you think? Are there any specific aspects and parts of the PR that you would like to have reviewed?

I have not really reviewed this, so it is still in need of reviewing.

Lord-McSweeney · 2023-05-20T00:57:56Z

https://www.nothingtodo.co.uk/view/3136/who-needs-a-glove.html logs "Video has no decoded frame to render" with this PR.

Lord-McSweeney · 2023-05-31T01:16:12Z

Since DisplayObjects use instance allocators now, this needs to define an instance allocator on Video.

…_frame`. You can also specify a `tick_rate` in order to get ticks faster or slower than the stage frame rate. We also enable video decode and ticks on the netstream tests, since our netstream impl requires ticks.

…e allocator function

…re the actual video data.

… remove them

…atible-ish datastream so that we can still play the video

torokati44 · 2023-06-30T08:37:07Z

I posted a different Screen V1 test FLV on Discord, which will hopefully be less sensitive to platform differences.

… display object are connected correctly

kmeisthax force-pushed the netstream-flv branch from 2832e76 to ee5c75e Compare April 21, 2023 21:40

kmeisthax force-pushed the netstream-flv branch from b6850c6 to fc16132 Compare April 22, 2023 16:02

kmeisthax marked this pull request as draft April 22, 2023 18:00

kmeisthax force-pushed the netstream-flv branch 3 times, most recently from 1a4e411 to eb4de1b Compare April 23, 2023 00:28

Lord-McSweeney reviewed Apr 29, 2023

View reviewed changes

core/src/avm2/globals/flash/media/video.rs Outdated Show resolved Hide resolved

This was referenced May 10, 2023

Issues on https://new.weedtowonder.org/domestication.html #10175

Open

Issues on https://new.weedtowonder.org/introduction.html #10173

Open

iwannabethedev mentioned this pull request May 12, 2023

Crash on neave_tv.swf #11007

Closed

iwannabethedev reviewed May 12, 2023

View reviewed changes

core/src/streams.rs Show resolved Hide resolved

iwannabethedev reviewed May 12, 2023

View reviewed changes

core/src/streams.rs Outdated Show resolved Hide resolved

iwannabethedev reviewed May 12, 2023

View reviewed changes

TemaisgameNoobas mentioned this pull request May 12, 2023

Add FLV support (there is a pull request for it) #11025

Closed

kmeisthax force-pushed the netstream-flv branch from eb4de1b to 1e5c2ac Compare June 1, 2023 01:21

kmeisthax added 16 commits June 29, 2023 19:35

tests: Add option to send ticks to the player instead of calling `run…

8fb260f

…_frame`. You can also specify a `tick_rate` in order to get ticks faster or slower than the stage frame rate. We also enable video decode and ticks on the netstream tests, since our netstream impl requires ticks.

docs: Document num_ticks and tick_rate

2b920d4

core: The stop event is NetStream.Play.Stop

a730595

core: The AVM2 version of this test wants a spurious status event

2fa8901

tests: Add images to the netstream tests

4429d0b

avm2: Video.init doesn't need to call super_init

1e02ed1

core: Log netstream signature verification errors

052ff95

core: Pause the stream if we hit a tag parsing error.

47e8877

avm2: Move as much Video initialization as possible into an instanc…

44502a2

…e allocator function

avm2: Use ParametersExt to grab parameters in Video.init()

8501495

avm2: Make netstream event initialization more concise

2f1a56a

flv: VP6 video packets have a horizontal and vertical adjustment befo…

fc5a4e7

…re the actual video data.

docs: Keep documentation of the old num_frames options until we can…

50e7da1

… remove them

flv: Screen Video wants the FLV VideoData header byte

f72ebca

flv: ScreenVideo v2 also wants the FLV video data byte

65c0e7f

core: If a video does not have an onMetaData block, register a comp…

2ba2c92

…atible-ish datastream so that we can still play the video

kmeisthax force-pushed the netstream-flv branch from 1f76912 to f7d14e1 Compare June 29, 2023 23:37

kmeisthax enabled auto-merge (rebase) June 29, 2023 23:42

tests: Add test of ScreenVideo FLVs (contributed by torokati44)

467adf9

kmeisthax force-pushed the netstream-flv branch from f7d14e1 to 925262e Compare June 30, 2023 03:34

core: AVM2 display object allocators always ensure both halves of the…

72541b1

… display object are connected correctly

kmeisthax force-pushed the netstream-flv branch from 925262e to 72541b1 Compare June 30, 2023 21:41

Merge branch 'master' into netstream-flv

6a04790

kmeisthax merged commit 32524d3 into ruffle-rs:master Jun 30, 2023

Lord-McSweeney mentioned this pull request Jul 1, 2023

AVM2 DO methods don't support Video DOs #10705

Closed

This was referenced Jul 1, 2023

Error on https://game1164.konggames.com/games/MateuszSkutnik/submachine-2-the-lighthouse/frame/61ddf6aa-48e9-4379-8b40-b9f12c1e6b58/ #11823

Closed

VP6 regression after merging FLV PR into master, broken ~150 NSFW games #11822

Closed

kmeisthax mentioned this pull request Jul 1, 2023

Fix post-FLV-merge regressions #11831

Merged

n0samu mentioned this pull request Jul 7, 2023

Micrics - Gamezhero Preloader not working #11380

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FLV support #10756

FLV support #10756

kmeisthax commented Apr 21, 2023 •

edited

Loading

TheGhostboi64 commented Apr 21, 2023

danielhjacobs commented Apr 21, 2023 •

edited

Loading

Lord-McSweeney commented Apr 21, 2023

waspennator commented Apr 21, 2023 •

edited

Loading

Lord-McSweeney commented Apr 21, 2023

danielhjacobs commented Apr 21, 2023

kmeisthax commented Apr 21, 2023

Lord-McSweeney commented Apr 22, 2023 •

edited

Loading

kmeisthax commented Apr 22, 2023

torokati44 commented Apr 22, 2023

iwannabethedev May 12, 2023

iwannabethedev May 12, 2023 •

edited

Loading

iwannabethedev May 12, 2023

iwannabethedev commented May 12, 2023 •

edited

Loading

Lord-McSweeney commented May 20, 2023

Lord-McSweeney commented May 31, 2023

torokati44 commented Jun 30, 2023

FLV support #10756

FLV support #10756

Conversation

kmeisthax commented Apr 21, 2023 • edited Loading

TheGhostboi64 commented Apr 21, 2023

danielhjacobs commented Apr 21, 2023 • edited Loading

Lord-McSweeney commented Apr 21, 2023

waspennator commented Apr 21, 2023 • edited Loading

Lord-McSweeney commented Apr 21, 2023

danielhjacobs commented Apr 21, 2023

kmeisthax commented Apr 21, 2023

Lord-McSweeney commented Apr 22, 2023 • edited Loading

kmeisthax commented Apr 22, 2023

torokati44 commented Apr 22, 2023

iwannabethedev May 12, 2023

Choose a reason for hiding this comment

iwannabethedev May 12, 2023 • edited Loading

Choose a reason for hiding this comment

iwannabethedev May 12, 2023

Choose a reason for hiding this comment

iwannabethedev commented May 12, 2023 • edited Loading

Lord-McSweeney commented May 20, 2023

Lord-McSweeney commented May 31, 2023

torokati44 commented Jun 30, 2023

kmeisthax commented Apr 21, 2023 •

edited

Loading

danielhjacobs commented Apr 21, 2023 •

edited

Loading

waspennator commented Apr 21, 2023 •

edited

Loading

Lord-McSweeney commented Apr 22, 2023 •

edited

Loading

iwannabethedev May 12, 2023 •

edited

Loading

iwannabethedev commented May 12, 2023 •

edited

Loading