Janus

Janus is a messaging server optimized to unicast over TCP to thousands of clients subscribed to topics of interest.

The ultimate goal is to maintain a latency of less than 2 seconds for 20 thousand clients on Amazon EC2 (small instance).

License

Janus is available under the MIT license.

Bounty and rules

1) Unicast to 20K clients in < 1s (max latency) nets you a $1000 bounty if you are the first person to achieve this. Unicast in < 2s (max latency) nets you a $500 bounty.

2) Final proof has to be from EC2, one small instance for the server and one for the bots.

3) {packet, 0}, -smp disabled

4) TCP, not UDP.

5) No change of the protocol unless I agree to it.

Note that SMP is disabled since Amazon EC2 small instances have a single core.

Installation

Janus requires at least OTP R13B01.

Create a soft link from janus/mochiweb to your Mochiweb installation.

Running

1) ‘make run1’ to start the server
2) ‘make sh’ in a different window to run clients
3) ‘bot:test(flashbot, 20000).’ to run 20k bots on the same machine
4) ‘bot:test(flashbot, 20000, ’host’, 8081).’ if the server is somewhere else

You should see output like this if everything goes well (MacBook Pro Core2Duo, 2.93Ghz, SSD):

(debug@biggie.local)5> bot:test(flashbot,10000).

=INFO REPORT==== 15-Jul-2009::19:44:20 ===
setup: 5055.70ms, n: 10000, run: 7524.29ms
    1.8690ms | min   
  500.0000ms | 2125   -  21.25%
 1000.0000ms | 5010   -  50.10%
 1500.0000ms | 2865   -  28.65%
 1269.7850ms | max   
ok
(debug@biggie.local)5> bot:test(flashbot,20000).

=INFO REPORT==== 15-Jul-2009::19:44:55 ===
setup: 14293.01ms, n: 20000, run: 21956.94ms
    2.4850ms | min   
  500.0000ms | 478    -   2.39%
 1000.0000ms | 2283   -  11.42%
 1500.0000ms | 7154   -  35.77%
 2000.0000ms | 1574   -   7.87%
 2500.0000ms | 3301   -  16.50%
 3000.0000ms | 2779   -  13.89%
 3500.0000ms | 2431   -  12.16%
 3277.0740ms | max   
ok

Notes

On Leopard:

cat /etc/sysctl.conf 
kern.maxfiles=102400
kern.maxfilesperproc=40960
net.inet.ip.portrange.hifirst=10000
net.inet.ip.portrange.first=10000

grep ulimit ~/.bash_profile
ulimit -n 30720 

Also, you won’t be able to go past 1024 clients with Erlang
on Mac OSX, even with kernel poll enabled (+K true).

Edit FD_SETSIZE in /usr/include/sys/select.h, i.e. change
__DARWIN_FD_SETSIZE in /usr/include/sys/_structs.h
and recompile Erlang with —enable-kernel-poll.

On Linux:

echo "8192 61000" > /proc/sys/net/ipv4/ip_local_port_range
echo "<your user name>             -       nofile          32768" >>
/etc/security/limits.conf
ulimit -n 32768