Overscan

A study in live video broadcasting with the Racket programming language.

The goal of this project is to learn Racket and to better understand the problems involved in:

• The Real-Time Messaging Protocol (RTMP) and constructing and streaming unbounded audio and video feeds.
• The Twitch and Facebook Live API's, for authentication, broadcasting live video streams, and gathering audience data.
• Video compositing in software.

Originally, the end-goal was to build a toolkit like that found in StreamPro or Streamlabs (née TwitchAlerts) but using a Racket DSL for on-the-fly compositing and graphics.

Now, this project's ambition is to provide a comprehensive live-coding environment for video compositing and broadcasting. This project is inspired by other live-coding environments like Impromptu/Extempore and Overtone.

It is split into three parts:

1. The ffi/unsafe/introspection module. This module provides dynamic Racket bindings to GObject Introspection, allowing interaction with C GObject libraries using common Racket idioms (i.e. providing racket/class forms such as send and get-field for GObjects).
2. The gstreamer collection. Using the aforementioned Introspection module, this collection provides Racket bindings for GStreamer, the open source multimedia framework.
3. The overscan collection and language. This provides a DSL for building a GStreamer pipeline for capturing common video sources (cameras and screens), compositing multiple sources, including generated graphics, and then encoding them and pushing them along to an RTMP server (like Twitch). All of this is designed to happen within a Racket REPL session, allowing the broadcaster full control over the stream by evaluating S-expressions. That's the idea, at least.

You can follow the development along at http://tinyletter.com/wunsch. Read archives of previous weekly devlogs at http://www.markwunsch.com/tinyletter/.

Installation

Overscan has been tested on macOS Sierra with Racket v6.12 and GStreamer v1.14.0. With a working Racket installation you can install with raco:

raco pkg install overscan

For Overscan to do anything meaningful, you also need to install GStreamer and necessary plugins. Using homebrew:

brew install gstreamer

This will install gstreamer along with dependencies glib and gobject-introspection, all of which are required.

You also need to install GStreamer plugins:

brew install gst-plugins-base --with-pango

brew install gst-plugins-good

brew install gst-plugins-bad --with-rtmpdump --with-fdk-aac

brew install gst-plugins-ugly --with-x264

Copyright © 2017 - 2018 Mark Wunsch

This package is distributed under the GNU Lesser General Public License (LGPL). This means that you can link overscan into proprietary applications, provided you follow the rules stated in the LGPL. You can also modify this package; if you distribute a modified version, you must distribute it under the terms of the LGPL, which in particular means that you must release the source code for the modified software. See http://www.gnu.org/copyleft/lesser.html for more information.