pwdownloader.sh, a semi-automated video downloader

Author

patrickwoolf

Importance

Download online videos in whole series semi-automatically from several mainstream video streamers online, e.g. 小鴨影音, Gimy劇迷, 貓狗影視, MOMOVOD (No longer supported), 楓林TV (no longer supported), ...:

Installation

• Change directory into the folder you want to save your videos.

$ cd $DIR                                                                                               

[← ] If you are a Windows user, you could download Ubuntu WSL from Microsoft Store, and run the commands below.

• 
• 
• Open Ubuntu from your application launcher (press Super/Windows and type in Ubuntu, open the terminal console)
• Download necessary packages from apt:

$ sudo apt install ffmpeg sed git

• Change directory to your windows system, for example, C:\Users\User\Videos\

$ cd /mnt/c/Users/User/Videos/

If you are a iOS user, you could download iSH from Testflight, and do the following command:

$ apk add git
$ git clone -b iOS https://github.com/patrickwoolf/m3u8-download.git
$ cd m3u8-download
$ ash setup.sh

And now you could jump to Usage.

• Clone directly from this repository

$ git clone https://github.com/patrickwoolf/m3u8-download.git
$ cd m3u8-download
$ bash setup.sh

Usage

If you want to download videos from YouTube...

$ bash pwdownloader.sh [YouTube url]

..., the above command works just fine.

Other sites

You can download from several video streamers online, e.g. Gimy劇迷, MOMOVOD, 楓林TV, ...

$ bash pwdownloader.sh [URL] # URL of whichever episode of the TV show will be fine, tye code will grab all epispdes available for you. Please note that URL of a overview page doesn't work.

You shall be prompted to choose from several m3u8 sources, please choose URLs from the same server. For example, if one show has 6 episodes in total, and there are 5 servers, then you may be asked to decide the start and end indices of servers (a total of 30). Please be reminded that your input should be like a b, e.g. 19 24.

Then you would be able to type in the video name you want, notice that the episode number would be added automatically later, it's not necessary to type the episodes as well. Check if the video name is correct... Check if the episodes are correct..., the bash script would be copied to a file named download-command.sh

After these process your videos will be downloaded in ./videos.

What does each script do?

• curl.sh grabs the source code from online video sites.
• curl_preprocessing.sh changes the m3u8 URLs curled from the website to human readable ones, which is then a text file curlresult.txt.
• m3u8-download.sh finalizes the prepare process. It reads your input server choice, on which curlresult.sh based. curlresult.sh is a list of command which will be parallely executed.
• parallel.sh is a script to assign multiple commands to PID, allowing parallel execution. Credit belongs to Alessandro Pezzato.
• m3u8-download is a script originally created by williamchanrico, using ffmpeg command to grab and combine .ts from m3u8 online stream.

Workarounds for DramaQ

Because for dramaq, grabbing m3u8 directly from curl may not be feasible, here's a sideway you can take to take advantage of the power of m3u8-download.

• First open Firefox and go to the video website of your interest, but do not play now.
• Open Network Monitor pressing shortcut F12, you should see the "Network" button in the toolbar.
• Go back to the video player, press F5 or whatever to reload the page. Now you shall play the embedded video.
• Go back to Network Monitor, now type "m3u8" (without ") in the filter on the upper left. You shall see something like index.m3u8 or *.m3u8 now.
• Right click on and copy the link, open the terminal on your computer:

  • Ubuntu/Debian and other Linux:

     $ bash ./scripts/m3u8-download [m3u8 url] [FILENAME]

  • MacOS

    • Download homebrew.
    • brew install ffmpeg.

     $ bash ./scripts/m3u8-download [m3u8 url] [FILENAME]

  • Windows

    Download WSL (Windows Subsystem for Linux) from Microsoft Store

    • 
    • Choose Ubuntu
    • 
    • Open Ubuntu from your application launcher (press Super/Windows and type in Ubuntu, open the terminal console)
    • download necessary packages from apt

     $ sudo apt install ffmpeg sed git

    • change directory to your windows system, for example, C:\Users\User\Videos\

     $ cd /mnt/c/Users/User/Videos/
     $ git clone https://github.com/patrickwoolf/m3u8-download.git
     $ cd m3u8-download/

    • Now you can use this repository smoothly.

     $ bash ./scripts/m3u8-download [URL] [FILENAME YOU WANT]

    • Whoa, your video would be downloaded now. Have a coffee, it will be finished in a few minutes based on the size of the video.

Acknowledgement

• Alessandro Pezzato on Stack Overflow
• williamchanrico/m3u8-download