Ishiki

Wrapper for Haibu and Http-Proxy

Please refer to Hachi for a simple API client module and command line tool.

What does it do?

It makes running a Node deployment server as painless as possible. It is currently aimed at people with single-server installations who intend to run a small development platform.

How does it work?

After starting Ishiki, an API will become available. With this API, you can deploy applications and manage them. If your application requires a specific version of Node, it will be set up automatically for you. Each application will run on its own IP:port internally, while being proxied through the domains specified on your app on whatever public port you want your sites to run on (e.g. 80).

Disclaimer

Ishiki is provided as is and, as it stands, should not be used for production.

Install

npm install haibu-ishiki

Usage:

node ./node_modules/haibu-ishiki/bin/ishiki

Or to install globally (preferred):

npm install haibu-ishiki -g

Usage:

ishiki

First time

When starting Ishiki for the first time, a default admin user will be created for you and a random password will be generated. Ishiki should output something along the lines of:

Initial admin account created:
> username: ishiki
> password: 12345667890abcdef

Make sure you take good note of the password (you can change it later).

Configuration

By default, Ishiki will run on the following settings:

{
  "host": "127.0.0.1",
  "port": "8080",
  "public-port": "80",
  "deploy-dir": "deployment",
  "port-range": {
    "min": "9080",
    "max": "10080"
  },
  "mongodb": {
    "host": "127.0.0.1",
    "port": "27017",
    "database": "ishiki"
  },
  "logs-size": 100000,
  "auth": {
    "active": true,
    "admin": "ishiki",
    "token_expiry": 1800
  },
  "haibu": {
    "env": "development",
    "advanced-replies": true,
    "useraccounts": true,
    "coffee": true,
    "directories": {
      "node-installs": "node-installs",
      "packages": "packages",
      "apps": "apps",
      "tmp": "tmp"
    }
  },
  "crossdomain": false
}

Specifying Config Settings

There is a config.sample.json file that you can modify with your own settings.

Inside the directory where haibu-ishiki is installed, you can copy the sample file to config.json:

ishiki

Or, you can use the -l or --local flag to load the config.json file relative to where you ran the command:

ishiki -l # Will load from ./config.json

Or, manage multiple config files by providing a filename to -l or --local:

ishiki -l config-prod.json # Will load from ./config-prod.json

Config Options

• host is the host Ishiki and its API will run on
• port is the port Ishiki and its API will run on
• public-port is the port the apps will be made available on to the public (proxy port)
• deploy-dir is where all the directories defined under haibu.directories go (defaults to <ishiki-dir>/deployment)
• port-range is the range of ports the apps will listen on internally before being proxied
• mongodb is the configuration for the MongoDB database
• logs-size is the cap on the log MongoDB collection where all the user/app logs go
• auth is for authentication. Set active to false to disable authentication, admin is the default admin username, token_expiry is the time in seconds a token can remain valid without activity (false for no expiry)
• haibu is whatever settings are available to the haibu module

Running Ishiki over HTTPS

If you specify https in your configuration file, Ishiki will automatically run over HTTPS instead of HTTP, e.g.:

"https": {
  "cert": "cert/ishiki.crt",
  "key": "cert/ishiki.key",
  "ca": "cert/ca.pem"
}

All subsequent calls to Ishiki's API will need to take this into account, with cURL for instance, add the -3 (or --sslv3) flag.

The ca (certificate authority) can be omitted if using a self-signed certificate. You will also need to run cURL with the -k (or --insecure) flag to ignore the verification.

API

With authentication turned on (default), all calls (except for /users/login) will need to explicitly specify a token in the URL, such as:

<http|https>://<ishiki-ip>:<ishiki-port>/<end-point>?token=<my-token>

The authentication token can be created with the help of /users/login

To enable CORS (for crossdomain usage), set crossdomain to true in your config.json.

Permissions

With the exception of logging in, permissions are as follow:

• users: admins can perform any action for any user, non-admins can only update their own password
• drones: admins can performs any action for any user, non-admins can only perform actions relating to their own drones (where /:userid is present)
• proxies: only admins may use this

Overview

• Users
• Get all users
• Create a new user
• Log-in (get an authentication token)
• Log-out
• Update user details
• Drones
• Get all drones
• Get drones for a given user
• Get drone for a given user and app
• Get running drones
• Deploy an app
• Start an app
• Stop an app
• Restart an app
• Get app logs
• Proxies
• Get all proxy routes
• Get all proxy routes for a given port
• Start a proxy on a new port
• Manually create a proxy route on a given port
• Stop proxy on a given port and clear its routes
• Delete proxy routes on a given port
• Get all proxy routes for a given user and port
• Get all proxy routes for a given app, user and port
• Delete routes for a given app, user and port

---

Users

/users (GET)

Returns a list of all users

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/users?token=<my-token>

Response

[ { "_id" : "51b12470b4a898d990000001",
    "admin" : true,
    "last_access" : "2013-06-08T22:47:22.828Z",
    "password" : "$2a$10$TtuNxZzX3bHdQSURpLLv4OHZ1QjbW2Fy6yRs3Cv1p6w414OnoOnTi",
    "token" : "d22b9961e33700436c76acfab2051ba73276b7fb5aa9e57bb1343fc9e5b1524f",
    "username" : "ishiki"
  } ]

---

/users (POST)

Creates a new user, if password is not provided, one will be generated. Set admin to true to give the new user admin rights.

Call example

curl -X POST -H 'Content-Type: application/json' -d '{"username": "myuser"}' <http|https>://<ishiki-ip>:<ishiki-port>/users?token=<my-token>

Response

{ "_id" : "51b390b90808e68d93000067",
  "admin" : false,
  "password" : "52360f1b10488ae7",
  "username" : "myuser"
}

---

/users/login (POST)

Returns an authentication token to be used for all other calls

Call example

curl -X POST -H 'Content-Type: application/json' -d '{"username": "myuser", "password": "mypassword"}' <http|https>://<ishiki-ip>:<ishiki-port>/users/login

Response

{ "token" : "f2623f7d089e58069caf123bda4eba614b30b67e20f90074bf7dfd6241e2e0e1" }

---

/users/logout (POST)

Revokes authentication token

Call example

curl -X POST <http|https>://<ishiki-ip>:<ishiki-port>/users/logout?token=<my-token>

Response

{ "message" : "You are no longer authenticated" }

---

/users/:userid (POST)

Updates a user, non-admin users can only update their own password, admins can update any details of any users with the exception of the username

Call example

curl -X POST -H 'Content-Type: application/json' -d '{"password": "mynewpassword"}' <http|https>://<ishiki-ip>:<ishiki-port>/users/myuser?token=<my-token>

Response

{ "message" : "Updated password" }

Drones

/drones (GET)

Returns a list of all drones

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/drones?token=<my-token>

Response

{ "drones" : [ { "_id" : "5104b15d936de54dd7000001",
        "dependencies" : { "express" : "3.0.6",
            "jade" : "*"
          },
        "directories" : { "home" : "user1-site1-1359269108855" },
        "domains" : "my.domain my.other.domain",
        "drones" : null,
        "engines" : { "node" : ">=0.8.0 <0.9.0" },
        "env" : { "PORT" : 9080 },
        "hash" : "811b9b44168d3cabcca54e67269f30a155da3f7e",
        "host" : "127.0.0.1",
        "name" : "site1",
        "private" : true,
        "repository" : { "directory" : "/home/bobthebuilder/node/haibu-ishiki/deployment/packages/user1-site1-1359269108855",
            "type" : "local"
          },
        "scripts" : { "start" : "app.js" },
        "started" : true,
        "uid" : "1QCm",
        "user" : "user1"
      },
    ] }

drones basically contains an array of the apps' package.json with a few added properties.

---

/drones/:userid (GET)

Returns all drones for a given user

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/drones/user1?token=<my-token>

Response

Same as /drones only with results being limited to specified user

---

/drones/:userid/:appid (GET)

Returns drone info for given user/app

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/drones/user1/site1?token=<my-token>

Response

Same as /drones only with results being limited to specified user and app

---

/drones/running (GET)

Returns all running drones

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/drones/running?token=<my-token>

Response

Same as /drones only with results being limited to drones that are started

---

/drones/:userid/:appid/deploy (POST)

Deploys an app from a tarball for given user/app, with Curl from your app's directory:

Call example

tar -cz . | curl -XPOST -m 360 -sSNT- <http|https>://<ishiki-ip>:<ishiki-port>/drones/user1/site1/deploy?token=<my-token>

Response

If anything goes wrong, an error will be returned, otherwise the raw drone's info will be returned. If the version of Node required by the new app being deployed isn't installed yet, it will be installed on the fly, which could very well mean that the query will timeout (hence -m 360 to allow to wait for 3 minutes). The installation of Node will keep going regardless and the application will be deployed right after. You can also check your application logs or the drones API to check the status of your drone.

---

/drones/:userid/:appid/start (POST)

Starts a previously stopped drone for given user/app

Call example

curl -X POST <http|https>://<ishiki-ip>:<ishiki-port>/drones/user1/site1/start?token=<my-token>

Response

{ "_id" : "5104b15d936de54dd7000001",
  "dependencies" : { "express" : "3.0.6",
      "jade" : "*"
    },
  "directories" : { "home" : "user1-site1-1359269108855" },
  "domains" : "my.domain my.other.domain",
  "drones" : null,
  "engines" : { "node" : "0.8.x" },
  "env" : { "PORT" : 9080 },
  "hash" : "811b9b44168d3cabcca54e67269f30a155da3f7e",
  "host" : "127.0.0.1",
  "name" : "site1",
  "port" : 9080,
  "private" : true,
  "repository" : { "directory" : "/home/bobthebuilder/node/haibu-ishiki/deployment/packages/user1-site1-1359269108855",
      "type" : "local"
    },
  "scripts" : { "start" : "app.js" },
  "started" : true,
  "uid" : "1QCm",
  "user" : "user1"
}

---

/drones/:userid/:appid/stop (POST)

Stops a running drone for given user/app

Call example

curl -X POST <http|https>://<ishiki-ip>:<ishiki-port>/drones/user1/site1/stop?token=<my-token>

Response

Same output as /drones/:userid/:appid/start with started set to false

---

/drones/:userid/:appid/restart (POST)

Restarts a running drone for given user/app

Call example

curl -X POST <http|https>://<ishiki-ip>:<ishiki-port>/drones/user1/site1/restart?token=<my-token>

Response

Same output as /drones/:userid/:appid/start

---

/drones/:userid/:appid/logs (GET)

Returns or streams the logs for a given app with optional filtering

Available filters

• type: 'info' or 'error' (defaults to both)
• limit: number of results (defaults to 10)
• stream: will create a stream on the log, essentially tail -f-ing it (changes output)

Call example - basic

curl -X GET -H 'Content-Type: application/json' -d '{"limit": 2}' <http|https>://<ishiki-ip>:<ishiki-port>/drones/user1/site1/logs?token=<my-token>

Response (JSON)

[ { "_id" : "5105ee9ee55d533f01000004",
    "app" : "site1",
    "msg" : "Express server started on port 1234\n",
    "ts" : "2013-01-28T03:21:02.498Z",
    "type" : "info",
    "user" : "user1"
  },
  { "_id" : "5105ee9de55d533f01000003",
    "app" : "site1",
    "msg" : "me so hungry\n",
    "ts" : "2013-01-28T03:21:01.499Z",
    "type" : "info",
    "user" : "user1"
  }
]

Call example - streaming

curl -X GET -H 'Content-Type: application/json' -d '{"stream": true}' <http|https>://<ishiki-ip>:<ishiki-port>/drones/user1/site1/logs?token=<my-token>

Response (plain text)

[Sun Jan 27 2013 22:19:57 GMT-0500 (EST)] [info] Express server started on port 1234
[Sun Jan 27 2013 22:19:58 GMT-0500 (EST)] [info] me so hungry
...

Proxy

/proxies (GET)

Returns a list of all proxies and associated routes

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/proxies?token=<my-token>

Response

{
  "80":{
    "test":{
      "host":"127.0.0.1",
      "port":9080,
      "user":"user1",
      "appid":"test"
    },
    "test.local":{
      "host":"127.0.0.1",
      "port":9080,
      "user":"user1",
      "appid":"test"
    }
  }
}

The top key is the port the proxy runs on, all the children are the internal target for the proxy for each domain.

---

/proxies/:port (GET)

Returns a list of all routes for proxy on given port

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/proxies/80?token=<my-token>

Response example

{
  "test":{
    "host":"127.0.0.1",
    "port":9080,
    "user":"user1",
    "appid":"test"
  },
  "test.local":{
    "host":"127.0.0.1",
    "port":9080,
    "user":"user1",
    "appid":"test"
  }
}

---

/proxies/:port (POST)

Starts a proxy on given port

Call example

curl -X POST <http|https>://<ishiki-ip>:<ishiki-port>/proxies/1234?token=<my-token>

Response

{"message":"Proxy started on 1234"}

---

/proxies/:port/set (POST)

Updates or creates an arbitrary route for proxy on given port with source domain and target host and port provided in POST. Routes created like this will be re-loaded when Ishiki restarts

Call example

curl -X POST -H 'Content-Type: application/json' -d '{"port": "12500","host": "internal.ip","domain": "my.domain"}' <http|https>://<ishiki-ip>:<ishiki-port>/proxies/80/set?token=<my-token>

Response

{"message":"Proxy route added: my.domain:80 > internal.ip:12500"}

---

/proxies/:port/delete_proxy (POST)

Stops and removes proxy and associated routes on given port

Call example

curl -X POST <http|https>://<ishiki-ip>:<ishiki-port>/proxies/1234/delete_proxy?token=<my-token>

Response

{"message":"Proxy no longer running on 1234"}

---

/proxies/:port/delete_route (POST)

If domain is provided in POST, corresponding route will be removed from proxy on given port. If domain is not provided, then all routes matching the contents of POST for proxy on given port will be deleted. In this case POST can have any of the following values for matching:

{
  "host": "1.2.3.4",
  "port": 1234,
  "user": "myuser",
  "appid": "myapp"
}

Call example

curl -X POST -H 'Content-Type: application/json' -d '{"domain":"my.domain"}' <http|https>://<ishiki-ip>:<ishiki-port>/proxies/1234/delete_route?token=<my-token>

Response

{"message":"Route deleted for my.domain on port 1234"}

---

/proxies/:port/:userid (GET)

Returns all routes for given user for proxy on given port

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/proxies/80/user1?token=<my-token>

Response

{
  "site1.com":{
    "host":"127.0.0.1",
    "port":9080,
    "user":"user1",
    "appid":"site1"
  },
  "site2.com":{
    "host":"127.0.0.1",
    "port":9081,
    "user":"user1",
    "appid":"site2"
  }
}

---

/proxies/:port/:userid/:appid (GET)

Returns all routes for given user/app for proxy on given port

Call example

curl -X GET <http|https>://<ishiki-ip>:<ishiki-port>/proxies/80/user1/site1?token=<my-token>

Response

{
  "site1.com":{
    "host":"127.0.0.1",
    "port":9080,
    "user":"user1",
    "appid":"site1"
  }
}

---

/proxies/:port/:userid/:appid/delete (POST)

Deletes route for given user/app for proxy on given port

Call example

curl -X POST <http|https>://<ishiki-ip>:<ishiki-port>/proxies/80/user1/site1/delete?token=<my-token>

Response

{"message":"Deleted routes for user user1, app site1 on port 80"}

Things to check before deploying an app

Domains

In your app's package.json, make sure you have at least one of domain, domains, subdomain and subdomains. These can be arrays or space-separated list of domains. These are used for proxy mapping. If not specified you can always set a new proxy route manually using the API.

Port

When your app runs a server, make sure it listens on process.env.PORT. While it still works without, it ensures that Ishiki will use one of the ports within the proxy port range defined in your configuration.

Dependencies

• union (0.3.6)
• flatiron (0.3.3)
• haibu (0.10.1)
• semver (1.1.2)
• tar (0.1.17)
• http-proxy (0.10.2)
• mongodb (1.2.x)
• bcrypt (0.7.x)
• commander (1.3.0)

Requirements

• Node 0.8.x
• MongoDB database
• root access (or sufficient rights) to run configure && make && make install to install Node versions

To Do

• drone start/stop/restart API
• automatically restart drones on server start
• add user authentication and permissions
• allow for persistent proxy routes
• user/app logs and log streaming
• tighten security for each user/drone
• create an easy-to-use client NPM module to work with Ishiki
• create a client interface for the browser

License

MIT License