Ultra HFR support (real time 240fps, 360fps, 480fps on true 240Hz, 360Hz, 480Hz displays)

[mdrejhon]

Hello -- I'm the founder of Blur Busters and the creator of the world's most popular motion tests called Blur Busters TestUFO (www.testufo.com) which is commonly used by owners of high-Hz monitors.

Anybody who sees the UFO icon on a reviewer website, they're using the free tests that I invented -- very popular among high-Hz gamers.

I am looking for tools to help create Ultra HFR content (real time 240fps, 480fps and 1000fps on real-time 240Hz, 480Hz and 1000Hz displays). Traditionally, the method was to use a slo-mo camera, and speed up the footage using instructions in the Ultra HFR FAQ

I tried to research the DAIN tool to see if it will support UltraHFR, e.g. converting 24fps to 240fps video files (for 240Hz monitors), or 60fps to 360fps video files (for 360Hz monitors), but documentation prominently shows 60fps and 60Hz,

Doubling Hz halves motion blur without needing black frames (or strobing like a CRT), so Ultra HFR has had some recent buss, especially since the new Christie Digital Cinema Projector now supports 480fps 480Hz operation. If you've seen a 120Hz iPad, or played at 120Hz at full frame rate for an extended time period -- you already have seen the benefits.

I have an ASUS 360 Hz PG259QN gaming monitor, so I want to create test 240fps and 360fps video files out of existing 30fps and 60fps video recordings I have here, with far better quality than classic interpolation.

I wrote an article called Frame Rate Amplification Technology and I consider your DAIN within this universe. New improvements have occured as of late, so I'm writing a new article later this year to mention DAIN in a sequel to this article too.

BONUS feature request: Add support for DAIN to convert existing slo-mo files to real-time motion? For example converting a 120fps slo-mo file (in a 30fps file), and output at a custom frame rate (e.g. 240fps) for output to a 240Hz monitor. That way, the ffmpeg step can be skipped. I'm willing to do the ffmpeg step first though, as long as there is support for ultra-high output frame rates.

[iBobbyTS]

By removing the rectify process in the network can speed up the whole process a lot.
If I'm interpolating 1 frame between 2 1920x1080 frames on a Tesla V100, it takes 40s with rectify and 0.2s without it.
This is a process that doesn't improve the result that much but takes a very long period of time. I think I'm using it correctly, if someone found this process will do a job please correct me.
I'm still looking for potential actions I can take to speed it up without too much loss of the algorithm.

[mdrejhon]

Oh, just to be clear.
There are two useful use cases:

1. "real-time" parlance referring to non-slow-motion UltraHFR video, that is pre-processed
2. "real-time" parlance referring to real-time processing

I presume DAIN is capable of item 1, since obviously, it is extremely slow at this time, but more artifactless. I recently received instructions (elsewhere) how to use DAIN to pre-generate real-time 240fps+ files from non-240fps files.

Project RIFE sounds useful for item 2

[mirh]

Reportedly new RIFE not only does support arbitrary interpolation factors, but it should also be top of its class as far as both speed and quality are concerned.