ynetd

Micromanage your microservices with a tiny super server written in go.

ynetd -proxy ":8080 localhost:8081" cmd arg...

• Listen on the specified address for connections.
• When one is received, launch the specified program (with arguments).
• Wait for the specified port to become ready.
• Forward this connection (and all future traffic) to the new service.

If you have a service that might not be used you can make it available while only consuming a tiny amount of RAM when it isn't needed.

The impetus for this was a docker-compose project with a handful of micro services. Each service consumes several hundred megabytes of RAM, while some of them might never be used during any given docker-compose up session.

Rather than fiddling with which services to launch or exclude and how to make them available when needed, just put ynetd in front of them. RAM usage for each container is now only 500K but the services will vivify automatically if a request to them is made.

Features

• Lazily load a service and forward all connections.

  Since it starts quickly, you don't need to configure clients to check and see if the port is listening, just connect and it will hold the connection until the service is ready.

• Stop the service after a period of inactivity.

  When the last client disconnects, wait the specified period of time, and shut it down. Reclaim those resources when you're done!

• Watch for the service to exit and restart it next time it is needed.

  This is additionally handy for services that crash, whether they throw exceptions, or are sacrificed by the OOM killer. No manual restarts (or inifinite loop wrappers) required.

• Wait longer for a service to start.

  Some services may open their port before really being ready to use (especially if they have their own child services). Configure an additional period to wait after starting the service before forwarding any traffic. Clients don't need to know they have to wait, they can connect instantly and the traffic will forward when ready.

• Forward multiple ports.

  If a service listens on multiple ports, it can listen to and forward all of them.

• Forward port traffic to another host.

  It can be run without a command specified and will simply forward the traffic. This can be a simpler alternative to using iptables or netsh since it requires no additional dependencies or special privileges and doesn't involve permanent system configuration.

Installation

Homebrew

Install via homebrew:

brew tap rwstauner/ynetd
brew install ynetd

Universal

Download the zip file for your OS from the releases page or use something like this:

YNETD_VERSION=v0.14
wget -qO /tmp/ynetd.zip https://github.com/rwstauner/ynetd/releases/download/$YNETD_VERSION/ynetd-linux-amd64.zip \
  && unzip -d /usr/local/bin/ /tmp/ynetd.zip \
  && rm -f /tmp/ynetd.zip

To verify the signature of the zip files:

YNETD_VERSION=v0.14
url=https://github.com/rwstauner/ynetd/releases/download/$YNETD_VERSION/ynetd-linux-amd64.zip \
  && wget -qO /tmp/ynetd.zip $url \
  && wget -qO /tmp/ynetd.zip.asc $url.asc \
  && gpg --no-tty --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 9791707D75D1474B6936CA216AD6ED6EA9371AED \
  && gpg --no-tty --batch --verify /tmp/ynetd.zip.asc /tmp/ynetd.zip \
  && unzip -d /usr/local/bin/ /tmp/ynetd.zip \
  && rm -f /tmp/ynetd.zip /tmp/ynetd.zip.asc

Configuration

One service can be configured on the command line:

-auto-start       # Start the command immediately
-proxy            # Address pairs to listen on/proxy to ("from:port to:port from2 to2")
-proxy-sep        # Alternate character to separate proxy addresses
-stop-after       # Duration after last connection to signal command to stop
-stop-signal      # Signal to send to command (default is TERM)
-timeout          # Duration to wait before aborting connection
-wait-after-start # Duration to wait after starting command before forwarding connections
args...           # Command to run

Alternatively (or additionally) multiple services can be configured by specifying the path to a YAML configuration file:

-config /path/to/ynetd.conf

Which should look something like:

---
services:
  -
    auto_start: true
    proxy:
      ":5000": "localhost:5001"
      "localhost:6543": "localhost:7654"
    command:
      - run
      - some
      - service
    stop_after: 10m
    stop_signal: INT
    timeout: 10s
    wait_after_start: 2s

Everything is optional (except for "proxy").

Without a command to run, ynetd can be used as a simple port forwarder:

ynetd -proxy "localhost:5001 remote.host:8080"

Addresses can be specified as:

• "host:port"

• ":port" (all addresses) (source)

• "interface:name:port" ("interface:eth0:5001") to listen on (all addresses of) the named interface (source)

This can be useful if you don't know the ip address up front but want to forward from public interface to loopback to avoid having to use additional port numbers:

-proxy "interface:eth0:5000 localhost:5000"

• "exec:/full/path/to/command" (destination)

Useful to get the destination address dynamically from a command. The command will receive the following arguments:

• local address (host:port)

• remote address (host:port) The command should print the destination "host:port" on stdout and return zero. If the command exits non-zero the status and stderr of the command will be printed to stdout and the connection will be dropped.

  -proxy "localhost:5000 exec:/usr/local/bin/current-tunnel-address"

NOTE: You must specify a full path (must start with a slash).

Why?

• To reduce resource utilization for services until they are needed.
• To regain resources when they are no longer needed.
• To simplify container configuration.
• To learn a little go.

Why the name?

I wanted something like xinetd but something that would launch another (long running) server and just forward connections.

• "y" comes after "x"
• Maybe [tin]ynetd
• Stop asking ":why: are you eating all my RAM?"
• y not?

License

Copyright © 2017 Randy Stauner. Distributed under the MIT License.