zImageOptimizer

Simple bash script for lossless image optimizing JPEG, PNG and GIF images in a specified directory include subdirectories on Linux, MacOS and FreeBSD.

Features

• lossless image optimization with a small image size in the output;
• works recursively;
• checks optimization tools after the start;
• option for automatic install dependencies and optimization tools if one or more of it is not found (see supported distributions here);
• readable output info and total info after optimization;
• no limit for a file size (limited only by hardware);
• no limit for a number of files;
• no limit for a length of a filename (limited only by a file system);
• supports for special characters (except slashes and back slashes), spaces, not latin characters in a filename;
• supports for search of images changed in a certain period of time;
• supports for use of the special time marker for search only new files (based on last modify time).
• supports extensions via special hooks

Requirements

• bash 4+

Tools

JPEG:

• jpegoptim
• jpegtran, djpeg and cjpeg (from libjpeg library)

PNG:

• pngcrush (v1.7.22+)
• optipng (v0.7+)
• pngout
• advpng (from AdvanceCOMP library)

GIF:

• gifsicle

One or more tools required for optimization. djpeg/cjpeg does not support CMYK colorspace and does not participate in the optimization of such images.

Notice: Optimization via djpeg/cjpeg is currently temporarily disabled to prevent an increase in size of output files in lossless mode. Later, djpeg/cjpeg options for lossless compression and small output file size will be selected.

Usage

Usual usage

bash zImageOptimizer.sh -p /path/to/files

or

bash zImageOptimizer.sh --path=/path/to/files

Supported options:

• -h (--help) - shows a help message;
• -v (--version) - shows a script version;
• -p (--path) - specify a full path to the input directory (usage: -p <dir> | --path=<dir>);
• -q (--quiet) - executes the script without any questions and users actions;
• -l (--less) - don't shows the optimizing process;
• -c (--check-only) - checks tools with an opportunity to install dependencies;
• -t (--time) - sets the period for which to look for files by last modified time (usage: -t <period> | --time=<period>);
• -n (--new-only) - use the time marker file for looking new images only;
• -m (--time-marker) - sets the custom full path or the custom filename of a time marker file (usage: -m <name|path> | --time-marker=<name|path>);
• -tmp (--tmp-path) - sets the custom directory full path for temporary files (usage: -tmp <dir> | --tmp-path=<dir>);
• -e (--exclude) - a comma separated parts list of paths to files for an exclusion from search (the script removes from the search files in the full path of which includes any value from the list; usage: -e <list> | --exclude=<list>);
• -ext (--extensions) - a comma separated list of script's extensions to enable. Script's extensions disabled by default. Use all to enable all found extensions (usage: -ext <list> | --extensions=<list> | -ext all | --extensions=all).
• --unlock - manually delete target dir from lockfile if previous script launch was interrupted incorrectly or killed by system.

Notices:

• you may combine options;
• -h (--help) option ignore all other options;
• -v (--version) option ignore all other options (except for -h (--help));
• -c (--check-only) option ignore all other options (except for -h (--help) and -v (--version));
• path in -p (--path) and -tmp (--tmp-path) options may be with and without a slash in the end of the path;
• it is impossible to use together -t (--time) and -n (--new-only) options;
• you must use -m (--time-marker) option with -n (--new-only) option;
• you must use --unlock option with -p (--path) option (just add --unlock option in next run);
• you can install dependencies without any questions in check mode (-c (--check-only)) using -q (--quiet) option.

Recommendation: use GNU Screen or analogs if there are many images in an input directory, because the optimization may can take long time.

After starting optimization, the script creates special temporary lock file (/tmp/zio.lock by default), where path to working directory is added. After optimization is finished, the script deletes this file (or deletes current path to working directory from the file with several parallel optimizations). This is done to prevent cycling optimization and avoid conflicts during optimization for longer than period between optimizations. Notice: if the script is terminated abnormally, you should delete lock file manually.

Excluding folders/files from search

bash zImageOptimizer.sh -p /path/to/files -e <list>

Example:

bash zImageOptimizer.sh -p /path/to/files -e /var/www/test.com,backup,uploads/orig.png

Usage with set the period

bash zImageOptimizer.sh -p /path/to/files -t <period>

Supported periods:

• minutes (10m, 30m etc.),
• hours (1h, 10h etc.),
• days (1d, 30d etc.).

Example:

bash zImageOptimizer.sh -p /path/to/files -t 15d

Usage with the time marker (recommended for a cron usage)

bash zImageOptimizer.sh -p /path/to/files -n

Notice: by default the time marker file created in working a directory which set in -p (--path) option with a filename .timeMarker.

Custom time marker name

Use -m (--time-marker) option and set a new filename if you want to change the time marker filename:

bash zImageOptimizer.sh -p /path/to/files -n -m myCustomMarkerName

Path to time marker will be /path/to/files/myCustomMarkerName

Custom time marker path and name

Use -m (--time-marker) option and set a new path and filename if you want to change the time marker path:

bash zImageOptimizer.sh -p /path/to/files -n -m /path/to/marker/directory/markerName

Path to the time marker will be /path/to/marker/directory/markerName

Cron usage

Using default time marker:

0 0 * * * /bin/bash zImageOptimizer.sh -p /first/directory -q -n
0 1 * * * /bin/bash zImageOptimizer.sh -p /second/directory -q -n

Using custom time marker path and filename:

0 0 * * * /bin/bash zImageOptimizer.sh -p /first/directory -q -n -m /path/to/first/marker/firstMarkerName
0 1 * * * /bin/bash zImageOptimizer.sh -p /second/directory -q -n -m /path/to/second/marker/secondMarkerName

Also you may collect all markers in own directory:

0 0 * * * /bin/bash zImageOptimizer.sh -p /first/directory -q -n -m /path/to/markers/directory/firstMarkerName
0 1 * * * /bin/bash zImageOptimizer.sh -p /second/directory -q -n -m /path/to/markers/directory/secondMarkerName

Notice: use -l (--less) option if you want exclude optimizing process in cron email messages

Manually create/modify the time marker file

You may manually create the time marker file or change it last modified time:

touch -m /path/to/marker/markerName

If you want to create the marker with specify time or change marker last modified time with specify time:

touch -t [[CC]YY]MMDDhhmm[.SS]

where:

• CC – 2 first digits of the year,
• YY – 2 last digits of the year,
• MM – month,
• DD – date,
• hh – hours in 24 format,
• mm – minutes,
• SS – seconds.

Example:

touch -t 201712041426.00 /path/to/marker/markerName

Usage with custom path to the temporary files directory

bash zImageOptimizer.sh -p /path/to/files -tmp /custom/path/to/temporary/directory

Docker

Docker image provided by Varun Batra.

If you have built the docker container then dependencies are installed already. To use docker either start with mounting to /work/images folder which is compressed every 15 days for example:

docker run -u "$UID:$GID" --volume /path/to/images:/work/images -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro  varunbatrait/zimageoptimizer

Or you can come up with your own CMD for example:

# mkdir /mnt/ImagesHundred/marker
docker run -it -u "$UID:$GID" -d --volume /mnt/ImagesHundred/marker:/work/marker --volume /mnt/ImagesHundred/images/:/work/images/ -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro varunbatrait/zimageoptimizer ./zImageOptimizer.sh -p /work/images/ -q -n -m /work/marker/marker

Automatical installing dependencies

Notice: curent user must be root or user with sudo access.

Start the script in the optimization mode (-p|--path) or the checking tools mode (-c|--check-only, recommended) if you want to install dependencies automatically. It will check installed tools and printing a choise option dialog if one or more tools not found. Select Install dependencies and exit option by typing a appropriate number and press enter. The script will install dependencies based on your platform, distribution and package manager. You may need to enter a password and confirm actions during installation dependencies. Restart script to recheck installed tools after that.

Supported on:

• DEB-based linux distributions (i686/amd64)
  • Debian 7+
  • Ubuntu 14.04+
• RPM-based linux distributions (i686/x86_64)
  • RHEL 6+
  • CentOS 6+
  • Fedora 24+
• FreeBSD 10+ (i686/amd64)
• MacOS 10+

Tested on:

• DEB-based linux distributions
  • Debian 7.11 i686 minimal
  • Debian 8.9 i686 minimal
  • Debian 9.2 amd64
  • Debian 10 i686
  • Debian 10 amd64
  • Debian 11 amd64
  • Debian 12 amd64
  • Ubuntu 14.04.5 amd64
  • Ubuntu 16.04.3 amd64
  • Ubuntu 18.04.2 amd64
• RPM-based linux distributions
  • RHEL 6.9 i686 minimal
  • RHEL 7.4 x86_64 server
  • CentOS 6.9 x86_64 minimal
  • CentOS 7.4.1708 x86_64 minimal
  • Fedora 24 i686 minimal
  • Fedora 25 x86_64 minimal
  • Fedora 26 x86_64 minimal
  • Fedora 27 x86_64 workstation
  • Fedora 30 beta x86_64 server (required initscripts package)
• FreeBSD (in some cases it may be necessary to run auto-install twice)
  • 10.3 i686
  • 11.1 amd64
  • 11.2 amd64
  • 12.0 amd64
• MacOS
  • 10.10
  • 10.11.6
  • 10.13.6
  • 11.2.1

If you have errors during installing dependencies on supported platforms please contact me or open issue.

Manual installing dependencies

Notice: curent user must be root or user with sudo access.

Install following packages from repositories/ports

DEB-based:

apt-get install jpegoptim libjpeg-progs pngcrush optipng advancecomp gifsicle wget autoconf automake libtool make bc -y

RHEL:

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(rpm -E '%{rhel}').noarch.rpm -y
subscription-manager repos --enable rhel-$(rpm -E '%{rhel}')-server-optional-rpms
yum install jpegoptim libjpeg* pngcrush optipng advancecomp gifsicle wget autoconf automake libtool make bc -y

CentOS:

yum install epel-release -y
yum install jpegoptim libjpeg* pngcrush optipng advancecomp gifsicle wget autoconf automake libtool make bc -y

Fedora:

dnf install epel-release -y
dnf install jpegoptim libjpeg* pngcrush optipng advancecomp gifsicle wget autoconf automake libtool make bc -y

MacOS:

Install homebrew if not installed

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install packages via homebrew

brew install jpegoptim libjpeg pngcrush optipng advancecomp gifsicle jonof/kenutils/pngout

Install new bash version (default 3.* version not supported) via homebrew

brew install bash

Add path to new bash version in /private/etc/shells

/usr/local/bin/bash

Add alias to new bash version in ~/.bash_profile

alias bash="/usr/local/bin/bash"

Logout and login again for enable new alias or add permanent alias for curent session

alias bash="/usr/local/bin/bash"

FreeBSD:

Install following ports

wget (/usr/ports/ftp/wget)
jpegoptim (/usr/ports/graphics/jpegoptim)
jpeg (/usr/ports/graphics/jpeg)
pngcrush (/usr/ports/graphics/pngcrush)
optipng (/usr/ports/graphics/optipng)
advancecomp (/usr/ports/archivers/advancecomp)
gifsicle (/usr/ports/graphics/gifsicle)

Install pngout

Linux:

wget http://static.jonof.id.au/dl/kenutils/pngout-20150319-linux.tar.gz
tar -xf pngout-20150319-linux.tar.gz
rm pngout-20150319-linux.tar.gz
cp pngout-20150319-linux/i686/pngout /bin/pngout     # for i686 arch
cp pngout-20150319-linux/x86_64/pngout /bin/pngout   # for x86_64/amd64 arch
rm -rf pngout-20150319-linux

FreeBSD:

wget http://static.jonof.id.au/dl/kenutils/pngout-20150319-bsd.tar.gz
tar -xf pngout-20150319-bsd.tar.gz
rm pngout-20150319-bsd.tar.gz
cp pngout-20150319-bsd/i686/pngout /bin/pngout    # for i686 arch
cp pngout-20150319-bsd/amd64/pngout /bin/pngout   # for amd64 arch
rm -rf pngout-20150319-bsd

Install pngcrush (RHEL/CentOS 6.*)

wget https://downloads.sourceforge.net/project/pmt/pngcrush/old-versions/1.8/1.8.0/pngcrush-1.8.0.tar.gz
tar -zxvf pngcrush-1.8.0.tar.gz
rm pngcrush-1.8.0.tar.gz
cd pngcrush-1.8.0
make
cp pngcrush /bin/
cd ../
rm -rf pngcrush-1.8.0

Install advpng (RHEL/CentOS 6.*)

yum install zlib-devel gcc-c++ -y
wget https://github.com/amadvance/advancecomp/releases/download/v2.0/advancecomp-2.0.tar.gz
tar -zxvf advancecomp-2.0.tar.gz
rm advancecomp-2.0.tar.gz
cd advancecomp-2.0
./configure
make
make install
cd ../
rm -rf advancecomp-2.0

Extensions

The script have support for extensions using special hooks. You can use prepared extensions or make own.

Extensions must be placed in extensions directory near the main script. Extensions disabled by default. Use -ext (--extensions) to enable needed:

bash zImageOptimizer.sh -p /path/to/files -ext my_extension,another_extension

Information about extensions you can see after launch the script. It automatically check selected extensions exists in extensions directory.

Or you can enable all extensions from dir extensions:

bash zImageOptimizer.sh -p /path/to/files -ext all

Troubleshooting

I'm install dependencies but one of tool is marked as NOT FOUND

By default the script looks for binary files into folowing directories /bin /usr/bin /usr/local/bin. If your binary file is not in these directories add your directory in variable BINARY_PATHS through a space like below and restart the script

BINARY_PATHS="/bin /usr/bin /usr/local/bin /your/custom/path"

I have errors djpeg: can't open /tmp/* and cjpeg: can't open /tmp/* during optimization

You have not write access to the directory /tmp. Tools djpeg and cjpeg use this directory for temporary files. Use -tmp (--tmp-path) option for set custom path.

I run the script but have error The directory is already locked by another script run! Exiting...

The script is already running in specified directory. If not, previous run of the script was not completed correctly. Delete dir from lock file (/tmp/zio.lock by default) or delete it manually and repeat. Also you can add --unlock option in next launch (dir automatically delete from lock file).

TODO

• add option for execute the script without any questions and users actions (for cron usage)
• add option for set time of the last change files for optimize only new images (for cron usage)
• add an option for set quality for more small files in the output
• add an option for check tools only
• add support for optimize gif images
• add support for automatic install dependencies on other platforms and distributions with other package managers
• add support for automatic install dependencies on others linux distributions
• add support for parallel optimization
• even more to improve results of compression
• add SVG support
• add WebP optimizing support (converting to WebP supported via extension)
• add support for detect wrong files extensions
• add support for convert or rename images with wrong files extensions
• add logging
• add Ansible playbook
• add progrees indicator
• add extensions support
• add lock file to prevent cyclical optimization with long image optimization
• add repository for debian/ubuntu
• add repository for rhel/centos/fedora

Contacts

• telegram @zevilz (EN|RU)
• telegram chat @zImageOptimizer (RU)

Reviews

• sysadmin.pm (RU)
• glashkoff.com (RU)

Donations

Do you like the script? Would you like to support its development? Feel free to donate

Changelog

• 14.09.2023 - 0.10.6 - check file exists before optimizing
• 09.03.2021 - 0.10.5 - added support for automatic install dependencies on MacOS 11+
• 27.09.2020 - 0.10.4 - fixed permissions and modify time for optimized images
• 03.09.2020 - 0.10.3 - preserve permissions and modify time during backup/restore
• 03.09.2020 - 0.10.2 - added BACKUP var in main loop for extensions, refactoring
• 16.08.2020 - 0.10.1 - bugfix
• 03.08.2020 - 0.10.0 - added --unlock option, added new hooks for extensions, bugfixes
• 04.05.2019 - 0.9.9 - bugfix (added check for using time marker in function fixTimeMarker)
• 02.05.2019 - 0.9.8 - changed logic of time marker, bugfix
• 02.05.2019 - 0.9.8 - changed logic of time marker, bugfix
• 29.04.2019 - 0.9.7 - improved support for extensions, added new hooks and more
• 20.04.2019 - 0.9.6 - added support for extensions, added lock file and more
• 06.02.2019 - 0.9.5 - fixed quality in cjpeg/djpeg compression
• 28.10.2018 - 0.9.4 - added check for empty temporary file after decompress images with djpeg
• 11.03.2018 - 0.9.3 - added a restoration of original files if their size is less or equal than optimized files size
• 27.02.2018 - 0.9.2 - fixed PNG file permissions if the script does not work from files owner
• 27.02.2018 - 0.9.1 - added new features and bugfixes
• 25.02.2018 - 0.9.0 - added support for working the script on MacOS 10.10+ with automatic install dependencies
• 24.02.2018 - 0.8.1 - added new parameter, updated info after optimizing, small fixes and small code refactoring
• 04.12.2017 - 0.8.0 - new features, bugfixes and code refactoring
• 30.11.2017 - 0.7.0 - added support for working the script on FreeBSD with automatic install dependencies, bug fixes and more
• 28.11.2017 - 0.6.0 - added support for automatic install dependencies on RHEL 6+ and Fedora 24+
• 25.11.2017 - 0.5.0 - bug fixes and code refactoring
• 25.11.2017 - 0.4.0 - added support for automatic install dependencies on Debian 7.* and some bugfixes
• 23.11.2017 - 0.3.0 - added support for automatic install dependencies on CentOS 6.*
• 22.11.2017 - 0.2.3 - some bug fixes
• 21.11.2017 - 0.2.2 - added support for automatic install dependencies on CentOS 7.*
• 20.11.2017 - 0.2.1 - some bug fixes
• 20.11.2017 - 0.2.0 - added some features and code refactoring
• 19.11.2017 - 0.1.1 - some bug fixes
• 19.11.2017 - 0.1.0 - beta released