This is a simple video-generator leveraging the wonders of free online stuff. It uses the GOES-R red, blue, and veggie bands to generate a near-realtime, true-color video of the past few days of weather.
With the most recent GOES satellites, NOAA has been posting near-realtime imagery to this Google Cloud bucket for public use.
Google also offers a very small VPS as a free demo, which I used to set the project up. See it live at bluemarble.nitk.in!
People usually discuss the frameworks they used, but this is pretty basic. It uses the latest in Javascript, HTML, and Python, with a bit of shell and make thrown in for good measure.
This code should be fairly easy to use - you'll need to modify paths to remove my username, but everything works!
The project consists of three largely independent components:
- Image processing
- Video generation
- HTTP server Cron acts as a job supervisor to hold everything together. (Again, quick and dirty).
GOES16.sci.py implements the image processing. When started, it goes into an infinite loop:
- Poll the Cloud Bucket API
- Check that all images are time-synchronized
- Check if images have changed
- If so, process into a JPG file
- If not, wait 5 minutes and try again.
Image processing is a fairly simple pipeline. For each layer (red, blue, veggie):
- Download the layer
- Convert from netCDF4 to netCDF3
- netCDF4 is newer, but the Python bindings for it are 5x slower than the netCDF3 libraries.
- nccopy is used for the conversion
- Scale to a managable size
- layers start as 10848 pixels square, which swamps the VPS
- Ensure all values are positive
- Negatives show up sometimes and mess up gamma correction
- Apply gamma correction
- Fancy word for taking x^0.55 for each pixel value
- Convert from numpy array to a PIL image
- Saves space and prepares for final formatting
Finally, the images are combined
- Create a green channel using red, blue, and veggie
- Stack the layers to make a composite
- Scale down to a thumbnail size for video-ing
The video generation is as dumb as a rock. cron periodically calls the video generator
with day/twoday/week/month/year as an argument.
The generator uses ffmpeg to build a video in /tmp; it errors out if the video
exists. (This prevents the generator from stepping on itself under high system loads)
Once generated, the video is moved from /tmp to DEST. Moves are atomic, which
ensures the file in DEST is always a valid video.
The length of the video and output filename are configured programatically by the FRAMES
dictionary. day, two-day, week, month, and year all do about what you'd expect.
Notably, month and year find the image nearest noon for each day. They
create a video with one frame per day, rather than six frames per hour.
The final component of the whole setup is a simple Python HTTP server. You could use Apache (or Nginx, or anything else), but on a disk-constrained server, I chose to use what was installed.
Compared to the standard Python server, it provides better support for If-Modified-Since
and threading.
It just hosts index.html, static.html and the videos.
The HTML file allows a user to select between the different generated videos, and automatically refreshes the videos to keep them near realtime.
Cron provides the glue for this project. The rules included periodically run the video generator,
and will automatically restart both the HTTP server and the image processor if either die.