Host Your Own Image Server based on MongoDB, Azure Blob Storage or Google Drive

Getting Started

Supported Platforms

• AWS S3 Storage

• Google Drive

• Azure Blob Storage

• MongoDB GridFS

Supported Formats

• image/jpeg
• image/png
• image/webp

💻 AWS S3 Storage

Setup steps for using AWS S3 Storage

If you have any public S3 url such as; https://[S3_NAME].s3.[S3_REGION].amazonaws.com/path/to/image you need to set these environment variables:

Name	Type
S3_NAME	string
S3_REGION	string

💻 Google Drive

You need to have a Google account to set up the project.

Setup steps for using Google Drive API

Firstly, create a new project from Google Developer Console. Go to the Credentials menu and create a new Service Accounts credential. It might be named as [PROJECT_ALIAS]-xxxx-xxxxxxx.json.

Download it and rename as gcloud-image-server-cred.json. Put the file to the root folder (it will be used for building Docker image).

Copy your service account mail address (it will be used for sharing your images with the project).

Setup steps for host the images

Go to your Drive page and create a folder, ie. image-server.

Share the folder with your service account (was copied before). Sharing rule will be applied to the all sub-items in the folder.

As a last step, upload any image (mime should be image/jpeg or image/png) to the folder and get the ID.

NOTE: ID is not showing anywhere in the portal, it sucks! To catch it, right click your item and select Get Shareable Link. Copy the last part of it, and toggle off the sharable link feature.

💻 Azure Blob Storage

Setup steps for using Azure Blob Storage

If you have an Azure Account and blob storage subscription, you have to create a new Access Key from portal or CLI. It could be like that; DefaultEndpointsProtocol=https;AccountName=[YOUR_STORAGE_NAME];AccountKey=[YOUR_ACCOUNT_KEY]==;EndpointSuffix=core.windows.net.

You need to add these values to the environment:

Name	Type
ABS_ACCOUNT_KEY	string
ABS_ACCOUNT_NAME	string
ABS_AZURE_URI	string

Setup steps for host the images

You can create a new container from Blob service > Container menus. Assume that you have a container which is named as sample-photos. It has two directories and a the picture at the last directory (summer > hotels > swimming.jpg).

Reach your blob with the url: http://127.0.0.1:8080/i/abs/100/400x0/sample-photos/summer/hotels/swimming.jpg

💻 MongoDB

Before running the application set these environment variables (or use Docker image, run with docker container run -d --name mongodb-instance -p 27017:27017 mongo:3.6.18-xenial and leave them default):

Name	Type	Default Value
MONGO_CONNECTION_STR	string	mongodb://127.0.0.1:27017
MONGO_DB_NAME	string	Photos
MONGO_MAX_POOL_SIZE	uint64	5

Run go run cmd/mongo-seed/main.go command and create 3 sample record on the DB such as;

_id	chunkSize	filename	length	metadata	uploadDate
5ec684803dd893bb72ead932	261120	image-3.jpg	710863	{"Content-Type": "image/jpeg"}	2020-05-21T13:39:12.585Z
5ec684803dd893bb72ead931	261120	image-2.jpg	516218	{"Content-Type": "image/jpeg"}	2020-05-21T13:39:12.603Z
5ec684803dd893bb72ead930	261120	image-1.jpg	379373	{"Content-Type": "image/jpeg"}	2020-05-21T13:39:12.617Z

Running

Via Docker

Build

# Build the image with Google Drive and WebP support. gcloud-image-server-cred.json file should be included!
docker image build -t [YOUR_REPOSITORY]/w-gdrive-w-webp:1.0.0 -f w-gdrive-w-webp.Dockerfile .
# Build the image with Google Drive without WebP support. gcloud-image-server-cred.json file should be included!
docker image build -t [YOUR_REPOSITORY]/w-gdrive-wo-webp:1.0.0 -f w-gdrive-wo-webp.Dockerfile .
# Build the image without Google Drive and with WebP support.
docker image build -t [YOUR_REPOSITORY]/wo-gdrive-w-webp:1.0.0 -f wo-gdrive-w-webp.Dockerfile .
# Build the image without Google Drive and withoutt WebP support.
docker image build -t [YOUR_REPOSITORY]/wo-gdrive-wo-webp:1.0.0 -f wo-gdrive-wo-webp.Dockerfile .

Run

docker container run -p 8080:8080 --rm [YOUR_REPOSITORY]/w-gdrive-w-webp:1.0.0
docker container run -p 8080:8080 --rm [YOUR_REPOSITORY]/w-gdrive-wo-webp:1.0.0
docker container run -p 8080:8080 --rm [YOUR_REPOSITORY]/wo-gdrive-w-webp:1.0.0
docker container run -p 8080:8080 --rm [YOUR_REPOSITORY]/wo-gdrive-wo-webp:1.0.0

Via Docker Compose

• docker-compose up

• Go to your favorite browser

• Surf to the URLs such as below.

Three containers will be bringing up. The first one is your image server application. It was written with purely the Go programming language. It can be reached with http://localhost:8080/version

Another one is Varnish HTTP Cache. It will automatically cache the output of the response for 14 days. It can be reached with http://localhost:8081/version

The latest one, nuster. Nuster, a high-performance HTTP proxy cache server and RESTful NoSQL cache server based on HAProxy. It will automatically cache the output of the response for 14 days like Varnish. It can be reached with http://localhost:8082/version

You can choose any cache server according to your experience.

Usage

/i/ endpoint is used for image operations, has four different usages:

AWS S3 Storage

/i/s3/webp/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{options:opt}/{*id}

/i/s3/webp/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{*id}

/i/s3/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{options:opt}/{*id}

/i/s3/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{*id}

Google Drive

/i/gdrive/webp/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{options:opt}/{*id}

/i/gdrive/webp/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{*id}

/i/gdrive/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{options:opt}/{*id}

/i/gdrive/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{*id}

Azure Blob Storage

/i/abs/webp/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{options:opt}/{*id}

/i/abs/webp/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{*id}

/i/abs/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{options:opt}/{*id}

/i/abs/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{*id}

MongoDB

/i/gridfs/webp/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{options:opt}/{*id}

/i/gridfs/webp/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{*id}

/i/gridfs/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{options:opt}/{*id}

/i/gridfs/{quality:range(0,100)}/{w:range(0,5000)}x{h:range(0,5000)}/{*id}

WebP Support

If you add /webp/ path to the URL, you can get the image as webp.

Options

Option	Description
g	This will convert the given image into a grayscale image.
t	This will scales the image up or down, crops it to the specified width and hight and returns the transformed image.
c	This will cuts out a rectangular region with the specified size from the center of the image and returns the cropped image.

Samples

• Original size: http://127.0.0.1:8080/i/s3/100/0x0/[PATH_TO_IMAGE], http://127.0.0.1:8080/i/gdrive/100/0x0/[YOUR_FILE_ID], http://127.0.0.1:8080/i/gridfs/100/0x0/[MONGODB_OBJECT_ID] or http://127.0.0.1:8080/i/abs/100/0x0/[YOUR_STORAGE_PATH]

• Resize with aspect ratio: http://127.0.0.1:8080/i/s3/100/500x0/[PATH_TO_IMAGE], http://127.0.0.1:8080/i/gdrive/100/500x0/[YOUR_FILE_ID], http://127.0.0.1:8080/i/gridfs/100/500x0/[MONGODB_OBJECT_ID] or http://127.0.0.1:8080/i/abs/100/500x0/[YOUR_STORAGE_PATH]

• Less quality: http://127.0.0.1:8080/i/s3/1/0x0/[PATH_TO_IMAGE], http://127.0.0.1:8080/i/gdrive/1/0x0/[YOUR_FILE_ID], http://127.0.0.1:8080/i/gridfs/1/0x0/[MONGODB_OBJECT_ID] or http://127.0.0.1:8080/i/abs/1/0x0/[YOUR_STORAGE_PATH]

• Resize without aspect ratio: http://127.0.0.1:8080/i/s3/100/1600x600/[PATH_TO_IMAGE], http://127.0.0.1:8080/i/gdrive/100/1600x600/[YOUR_FILE_ID], http://127.0.0.1:8080/i/gridfs/100/1600x600/[MONGODB_OBJECT_ID] or http://127.0.0.1:8080/i/abs/100/1600x600/[YOUR_STORAGE_PATH]

• Resize with crop: http://127.0.0.1:8080/i/s3/100/1600x600/c/[PATH_TO_IMAGE], http://127.0.0.1:8080/i/gdrive/100/1600x600/c/[YOUR_FILE_ID], http://127.0.0.1:8080/i/gridfs/100/1600x600/c/[MONGODB_OBJECT_ID] or http://127.0.0.1:8080/i/abs/100/1600x600/c/[YOUR_STORAGE_PATH]

• Create thumbnail with aspect ratio: http://127.0.0.1:8080/i/s3/100/0x300/t/[PATH_TO_IMAGE], http://127.0.0.1:8080/i/gdrive/100/0x300/t/[YOUR_FILE_ID], http://127.0.0.1:8080/i/gridfs/100/0x300/t/[MONGODB_OBJECT_ID] or http://127.0.0.1:8080/i/abs/100/0x300/t/[YOUR_STORAGE_PATH]

• Grayscale: http://127.0.0.1:8080/i/s3/100/900x0/g/[PATH_TO_IMAGE], http://127.0.0.1:8080/i/gdrive/100/900x0/g/[YOUR_FILE_ID], http://127.0.0.1:8080/i/gridfs/100/900x0/g/[MONGODB_OBJECT_ID] or http://127.0.0.1:8080/i/abs/100/900x0/g/[YOUR_STORAGE_PATH]

LICENSE

This project is licensed under the MIT License - see the LICENSE file for details.