Paraview: Combining Stills

Making a Movie

Assembling Frames into Movies

Using Scripts to Combine Images

There is a script called "CombineImages.sh" under spec/Support/Visualization/ParaviewPythonScripting that automatically combines one or two sets of images into a movie using ffmpeg. Note that this will count the files in the current directory, so there should not be anything in there besides the image files.

One Set of Images

  * Set GWdata in global variables to be 0
  * Make sure all your images are titled movietop.0001.jpg, movietop.0002.jpg, etc
  * Set OutputLocation in global variables to preferred location
  * Run script in location of images

Two Sets of Images (Creates a movie where each frame is composed of one image placed on top of another image)

  * Set GWdata in global variables to be 1
  * Make sure all your images that you wish to be on the top section of the movie be titled movietop.0001.jpg, movietop.0002.jpg, etc
  * Make sure all your images that you wish to be on the bottom section of the movie be titled moviebottom.0001.jpg, moviebottom.0002.jpg, etc
  * Set OutputLocation in global variables to preferred location
  * Run script in location of images

Using ffmpeg manually to Combine Images

Most visualization programs can output an animation as a sequence of frames in a common image format (PNG, JPEG). Using ffmpeg, these can be assembled into a movie with any framerate you like. Even better, you can use x264 to take total control over the encoding, ensuring optimal filesize and compatibility. Here is the procedure (assumes a modern Linux multimedia stack):

1. Output your animation as a sequence of frames. A lossless format like PNG is preferred. The frames should be numbered sequentially (myframe.0001.png, myframe.0002.png, ...).
   • In ParaView, go to File -> Save Animation. Choose your resolution and click 'Save Animation' (the framerate doesn't matter -- you'll choose it later). In the following dialog, choose "PNG images" as the filetype, create a folder to store the frames, and pick a filename prefix (like "myframe").
2. In a terminal, cd into the directory containing your frames and execute the following commands to create an MP4 video that is both small in size and compatible with mobile devices: ffmpeg -i 'myframe.%04d.png' -pix_fmt yuv420p -f yuv4mpegpipe - | x264 --crf 26 --preset veryslow --tune animation --fps 25 --keyint 75 --profile baseline --level 3.0 --vbv-maxrate 2500 --vbv-bufsize 2500 --ref 5 --output mymovie.264 --demuxer y4m - MP4Box -add mymovie.264:fps=25 mymovie.mp4
   • The input filename (myframe.%04d.png) uses printf format strings to indicate how your sequence of frames is numbered. This example is appropriate for use with ParaView.
   • Replace --fps 25 with the framerate you would like to movie to have (if you have very few frames, you can choose a small value here to have a slideshow-like effect).
   • The --keyint parameter controls the precision of seeking when playing back the video (at the expense of filesize). A value of triple the framerate allows the user to seek to within 3s, for example.
   • The quality is controlled by --crf 26. Lower numbers are higher quality.
   • For compatibility with mobile devices like smartphones (including the original iPhone):
     • You must not alter the line beginning with --profile baseline
     • Your resolution and frame rate must not exceed the limits of H.264 level 3. If you stay within 720x480@30fps, you'll be fine. This is best done when rendering the animation frames, but can also be done by adding something like "-s 640x360 -sws_flags spline" to the ffmpeg line.
   • Muxing with MP4Box produces a file compatible with streaming. (To remux an existing MP4 file, use MP4Box -inter 500 movie.mp4.)
   • If you have a recent version of x264, see Encoding Recipes in the Appendix for more advanced options

You can also combine images by executing the last line of the CombineImages.sh script. Make sure to replace the ${FrameRate} with a reasonable frame rate. This command should look something like: ffmpeg -b 4000k -r 50 -i movie.%04d.jpg Movie.mpg

Editing Movies with Kdenlive

Kdenlive is a free, open-source video editing software package which contains many useful features such as video effects, transitions, and title clips. It comes with myriad options, but the most useful and noteworthy are the Basics, Fades, and Titles.

Basics

Upon opening a new Kdenlive project, you will see 5 main sections:

1. 1 for your added video clips and their respective effects: Project Tree displays video clips which have been added and are ready to be inserted into the current project; Effect Stack displays all of the effects which have been added to the selected clip in the current project
2. 1 which lists available effects
3. 1 which displays selected clips/projects: Clip Monitor displays the selected clip in the Project Tree; Project Monitor displays the current project which is also represented chronologically by the tracks in the bottom section
4. 1 which lists all changes as they are made: Selecting a particular change from the list allows one to "undo" all subsequent changes made
5. 1 which houses the current tracks of video and audio

To add a new video/audio clip to your current Kdenlive project, right click in the window on the left which contains the Project Tree and select "Add Clip." Once a clip has been added to the Project Tree, you can add it to the current project by dragging the clip into a track in the bottom-most section.

To render a project, simply click the large "Render" button at the top of the page. This opens a window which designates output destination and allows you to choose from several export formats, most notably, H264, MPEG-2, MPEG-4, HDV, and Flash.

Fades

Fade to/from black are two of the more useful effects Kdenlive offers. Kdenlive automatically puts fade from black at the beginning of the selected clip and fade to black at the end. To add fades to a project:

1. Select the desired clip on its corresponding track at the bottom of the screen
2. Split the clip (if necessary) with the razor tool, represented by a pair of scissors at the bottom of the page

  *Splitting the clip allows you to put fades in, say, the middle of your project clip
- Drag fade to black, fade from black, or both effects from the effects list to the selected project clip

To adjust the duration of fades, do so in the Effect Stack of the selected clip, or change it manually by dragging the upper corner of the fade effect in the track display to the desired length.

Titles

To add a title clip to your current project, go to the Project Tree, right click, and select "Add Title Clip."

This opens a Title Clip window which allows you to customize your title before adding it to your project. This window has many features:

 *Add text by clicking the "T" icon in the top left; change font, size, and colors at the top; re-position with the arrowed buttons on the right
 *Add rectangles or images with the appropriate buttons located to the right of the text icon at the upper left of the window
 *Change the background or add animation using the tools under the corresponding tabs on the right of the window 

To create the title clip, click "Ok." To return to the Title Clip window after you have made a title clip, simply double click on the clip in the Project Tree.

Drag the newly created title clip from the Project Tree to an open track. Now, you can add video effects, such as fades, just as you would to any other clip.

More importantly, make sure you can see your title when you visualize your project. One solution is to have the title clip play before your other video clips (by dragging the other clips farther along the track timeline). However, if you want the title to overlap your existing video clip, do the following:

 *Click on the "Composite" box which appears under the Title clip in its track (this track also allows you to adjust the title's size/position)
 *In the Transition Window that appears in the top left, locate the "Composite" and "Auto" options at the top 
 *Change "Auto" to the track number which corrseponds to the video clip which will be overlapped by the title clip

Other Effects

Kdenlive offers many other video effects and transitions, all listed in the Effect List. To add an effect to your current project, simply drag the desired effect to the correct project clip track, and adjust the effect's settings in the Effect Stack window. The razor tool, a pair of scissors at the bottom of the page, is also useful in conjunction with effects as it allows you to cut clips.

Putting Movies Inside Other Movies

Sometimes, especially with complicated volume data, it can be nice to have more than one view in a movie simultaneously. Unfortunately, there is no way I know of to do this in visit or paraview, but the workaround is not terribly complicated.

• First render the movie in the main view you want, complete with annotations and everything. However, add two lines delineating the space in which you wish for the picture-in-picture to take place. This box should have the same aspect ratio as the images you are saving. Important: If using visit, make sure to save the files in the format name# (e.g the basename movie1, the files come out movie10001 movie 10002 etc.). This makes it incredibly easier to deal with the files in a script, as it is difficult to tell the computer the difference between movie00001 and movie1. I do not think you can render the image names as such in paraview, although I may be wrong. If not, you will either have to find a way to get around the above problem or simply rename them to something more convenient.
• Next, render the movie in the secondary view. Since this will be a small image in the final movie, it is probably a good idea to remove unnecessary annotations which would be too small to read and serve only to clutter the final product. Again, follow the naming scheme from above with a different leading digit, such as movie2.
• The composite command allows us to stitch these two images together in a coherent fasion. There is a (not very well documented or general) script for this in SpEC/Support/Visualization, but I will document the procedure here as well. The specific composite command to run is: composite smallimage.jpeg -gravity Southeast -geometry "59%" mainimage.jpeg compositeimage.jpeg The gravity and geometry parameters are specific to this case. Possible values for gravity indicate where the small image will be placed, and the geometry parameter determines the scaling.
• The other command in PictureInPicture.sh creates the small label in the picture in picture box. This command is: convert -font helvetica -fill black -pointsize 16 -draw 'text 830,665 "Edge-On View"' compositeimage.jpeg compositeimage-annotated.jpeg The parameters for this command are self-explanatory and should be edited for the specific movie. Of course, you could even annotate the movie in visit/paraview while making the primary image set as well.
• Finally, equipped with the composite images, simply make a movie out of them as you would any other set of images: convert -quality 100 composite-annotated* mymovie.mpeg

Converting Movies for the Website

Our public website, black-holes.org, can host movies of your simulations. But depending on how they were created, they may not be well-suited for the web for reasons of file size, resolution, and format/codec (if your file ends in .avi or .mpeg, it is absolutely not suited for the web). Fortunately, it is now easy to convert videos and create copies suitable for numerous purposes.

If you would like your movie embedded in the web site, it must meet the following requirements:

• H.264 codec, Baseline Profile, Level 3 (or iPhone preset if using a GUI)
  • Note that the Level 3 requirement places limits on resolution and framerate. Stay within 720x480@30fps and you'll be fine.
• MP4 container (M4V should be equivalent)

The website uses JW Player to embed videos, which defaults to using the HTML5 <video> tag and falls back on a Flash-based plugin in browsers that do not support the tag or codec.

In addition to embedding your video, you might also want to make high-resolution copies available for direct download. While these could be in any format you like (placing the burden on the end user), for reasons of file size and compatibility we highly recommend sticking to the H.264 codec, Main Profile, Level 3.1 in an MP4 container (this allows for 720p HD video compatible with newer smartphones and tablets).

The easiest way to convert your videos is probably to use the tool Handbrake. But if you prefer to do things by hand (I know you do), we recommend the x264 CLI. See Encoding Recipes in the Appendix for instructions.

** You must obey these format/codec constraints!** If you don't, you will break our web page for many viewers using a variety of mobile devices and web browsers.

Adding Movies to the Website

To make your movie available on the public website, follow these steps:

1. Create a web-safe, streamable version of your video, along with any other resolutions you would like to make public (see above)
2. Take a screenshot (or individual frame) from your web-safe movie and save it as a JPEG image (this is the picture a visitor will see before clicking on the movie to play it)
3. Choose a title for your movie, and write a description of the simulation that will be accessible to the public

At this point, if you do not have access to the web server, collect these files and bits of text in one place and contact Curran or Mike Boyle. If you do have access to the web server, read on:

1. Log on to the black-holes.org server, cd to /web/html/movies/, and mkdir a unique folder for your movie. Upload the video files and screenshot to this directory
2. Fill in the following template and add it to the web page you would like to embed this movie in:`... ... <script type="text/javascript" src="/jwplayer/jwplayer.js"></script> ...

   Movie Title

   Click on the picture below to watch the movie.

   <script type="text/javascript"> jwplayer("unique-id").setup({ skin: "/jwplayer/skins/glow/glow.zip", players: [ { type: "html5" }, { type: "flash", src: "/jwplayer/player.swf" } ] }); </script>

   Available for download in low resolution (640x360, M kB) and high resolution (1280x720, N MB).

   Description of the movie accessible to the public.

   ... `**Be sure to change `unique-id` to some unique string!** See `/web/html/explore2.html` for working examples (look near the bottom, not near the top). Contact [Curran](/mailto/curran@astro.cornell.edu) if you need assistance.

Appendix: x264 Flags

The world of digital multimedia is incredibly convoluted, and you do not want to waste as much time as Curran learning all of its dark secrets. But compatibility is a very important issue when it comes to online video, and it's easy to get wrong. Don't try to figure this out for yourself, and definitely don't trust an application to do it right for you (not without testing its output on a variety of mobile devices, operating systems, and web browsers). Curran has wasted enough time doing this; just follow his advice:

Software Requirements

The below procedures all use the command-line tool x264, and they require a relatively recent version of it. If you are using Ubuntu, you must be running version 11.10 or later. If these scripts don't work for you, talk to Curran about using his workstation "sherlock" for the task.

The following Ubuntu packages are required. If you are using another Linux distrubution, their names should be similar:

• x264 (for x264)
• gpac (for MP4Box)

Encoding Recipes

Some notes:

• In the following examples, the arguments to --keyint and --crf may be changed to meet the needs of your movie.
• Replace "video-in.avi" with the name of your original video
• Your original video must not have a framerate exceeding 30fps

Embedded Streaming / Smartphones

This is what you should use to create videos to embed in the black-holes.org website.

x264 --crf 26 --preset veryslow --tune animation --keyint 90 \
     --profile baseline --level 3.0 --vbv-maxrate 2500 --vbv-bufsize 2500 --ref 6 \
     --non-deterministic --vf resize:width=640,height=480,sar=1:1,fittobox=both,method=spline \
     --muxer mp4 --output video-out.mp4 --demuxer lavf video-in.avi
MP4Box -inter 500 video-out.mp4

Compatibility: iPhone 3G and beyond

HD Video / Tablets (720p)

x264 --crf 23 --preset veryslow --tune animation --keyint 90 \
     --profile main --level 3.1 --vbv-maxrate 14000 --vbv-bufsize 14000 \
     --non-deterministic --vf resize:width=1280,height=720,sar=1:1,fittobox=both,method=spline \
     --muxer mp4 --output video-out.mp4 --demuxer lavf video-in.avi
MP4Box -inter 500 video-out.mp4

Compatibility: iPhone 4, iPad and beyond

Full HD Video (1080p)

x264 --crf 20 --preset veryslow --tune film --keyint 90 \
     --profile high --level 4.1 --vbv-maxrate 50000 --vbv-bufsize 62500 \
     --non-deterministic --vf resize:width=1920,height=1200,sar=1:1,fittobox=both,method=spline \
     --muxer mp4 --output video-out.mp4 --demuxer lavf video-in.avi
MP4Box -inter 500 video-out.mp4

Compatibility: Powerful laptops and desktops with 1080p displays