realtime polyglot app node ruby mongodb socketio

リアルタイムのアプリ、または、プッシュベースのインタラクティビティを組み込んだイベント駆動型アプリは、 チャット、ラージスケールのゲーム、共同編集作業、低待ち時間での通知といった、ブラウザ内での動作の可能性を高める 新しい世代のベースとなります。たくさんのテクノロジーが、この流れをサポートしていますが、 以下の4つのテクノロジーが突出しています。: Ruby, Node.js, MongoDB そして Socket.IOです。

この記事では、これらの技術を使用したリアルタイムのアプリを構築するためのアーキテクチャとコードの両方を知ることが出来ます。

この記事のRuby data-writer componentのサンプルコードと Node.jsのWebアプリケーションはGitHub上で利用可能で、 動作に関しては、http://tractorpush.herokuapp.com/で確認可能です。

概要

サンプルのTractorPushリアルタイムアプリケーションは、MongoDbのキューにメッセージを格納するために Rubyベースのデータコンポーネントを使用し、次にそのキューがNode.jsのWebアプリに受信され、 Socket.IOを経由してユーザーのブラウザへ表示されます。 要するに全てのスタックは、プッシュ通知により動作します。

このシステムは、システム全体を作り上げるそれぞれの独立したアプリケーション同士を結びつける糊として動作する 後方サービス(MongoDB)を共有します。　

メッセージキューとしてのMongoDB

キューは、相互運用性があるものの独立したプロセスを記述するためにパワフルなメカニズムとなります。 商用的に実行可能なソリューションとして何百はなくとも、何十も存在しています。 古くから存在し、広く実装されているIBM WebSphere MQに始まり、 より新しくオープンソースかつオープンスタンダードなRabbitMQ や ZeroMQがあります。

MongoDBは、その柔軟なドキュメント保存の特性、多岐に渡る各言語へのサポート、 そしてtailable cursor の「プッシュ」の特徴により、 有用な多言語のメッセージキューとして役立ちます。 複雑で勝手気ままなJSONメッセージのマーシャリングとアンマーシャリングは自動的にハンドリングされます。 Safe-writeは、改良されたメッセージの耐久性と信頼性に対し効果があり、 tailable cursorは、MongoDBからNode.jsへの「プッシュ」を行うために使われます。

Provision Ruby publisher app

Ruby's mature web frameworks (Rails, Sinatra) make it ideal for the user-facing portion of most web apps which is often the origin of queued messages. TractorPush simulates this with a data-writer written in Ruby.

Create application

Clone the app from GitHub and create the app on Heroku.

:::term
$ git clone https://github.com/mongolab/tractorpush-writer-ruby.git
Cloning into tractorpush-writer-ruby...
...
Resolving deltas: 100% (8/8), done.

$ cd tractorpush-writer-ruby
$ heroku create tp-writer
Creating tp-writer... done, stack is cedar
http://tp-writer.herokuapp.com/ | git@heroku.com:tp-writer.git
Git remote heroku added

Attach MongoDB

Provision the MongoLab add-on to create the MongoDB instance that will contain the message queue.

:::term
$ heroku addons:add mongolab
----> Adding mongolab to tp-writer... done, v2 (free)
      Welcome to MongoLab.

Configure MongoDB capped collection

A MongoDB capped collection supports tailable cursors which allows MongoDB to push data to the listeners. If this type of cursor reaches the end of the result set, instead of returning with an exception, it blocks until new documents are inserted into the collection, returning the new document.

Also capped collections are extremely high performance. In fact, MongoDB internally uses a capped collection for storing the operations log (or oplog). As a trade-off, capped collections are fixed (i.e. “capped” in size) and not shardable. For many applications this is acceptable.

The TractorPush application relies on a capped collection in MongoDB to store messages. Open the MongoLab add-on dashboard for the writer app with heroku addons:open mongolab and add a new collection using the "Add" button.

Name the collection messages and expand the advanced options to specify a capped collection of 8,000,000 bytes (ample space for the demo).

Deploy publisher app

We have created the message writer app on Heroku and have configured the MongoLab add-on. Next, deploy the application to Heroku.

:::term
$ git push heroku master
Counting objects: 20, done.
...
-----> Heroku receiving push
-----> Ruby app detected
...
-----> Launching... done, v4
       http://tp-writer.herokuapp.com deployed to Heroku

The TractorPush writer is a simple and headless Ruby program that writes a message to the MongoDB database. Looking at the Procfile reveals that the writer process type is labeled worker. Scale the worker process to a single dyno to begin queuing messages.

:::term
$ heroku ps:scale worker=1
Scaling worker processes... done, now running 1

Verify that the worker process is running with heroku ps.

:::term
$ heroku ps
=== worker: `ruby writer.rb`
worker.1: up for 47s

Using Ruby to queue messages

The TractorPush application uses three types of document-based messages to demonstrate the flexibility of MongoDB object marshalling/unmarshalling: simple (or name-value), array and complex (or nested document) messages.

The writer.rb script writes one of the three document types to a MongoDB collection at a default rate of one per second.

:::ruby
while(true)
  coll.insert(doc, :safe => true)
  sleep(rate)
end

The :safe write-option ensures that the database has received and acknowledged the message document without error.

You can also access the MongoLab add-on dashboard with `heroku addons:open mongolab` to view the increase in the collection's contents and document count.

Viewing the logs shows the message types as they're queued.

:::term
$ heroku logs -t --ps worker.1
2012-03-23T14:56:35+00:00 app[worker.1]: Inserting complex message
2012-03-23T14:56:36+00:00 app[worker.1]: Inserting simple message
2012-03-23T14:56:37+00:00 app[worker.1]: Inserting simple message
2012-03-23T14:56:38+00:00 app[worker.1]: Inserting array message

Because the `messages` collection is a capped collection old documents will be discarded if the collection size exceeds its limit.

Provision Node.js web app

The consumer side of the system is a Node.js web application that consumes messages from the capped collection. Clone the app locally and create the app on Heroku.

:::term
$ git clone https://github.com/mongolab/tractorpush-server.git
Cloning into tractorpush-server...
...
Resolving deltas: 100% (8/8), done.

$ cd tractorpush-server
$ heroku create tp-web
Creating tp-web… done, stack is cedar
http://tp-web.herokuapp.com/ | git@heroku.com:tp-web.git
Git remote heroku added

Sharing application resources

In order to use two language environments (Node.js and Ruby) as a single system the two applications must share the message-store. Share the MongoDB instance between the writer and web apps by copying the MONGOLAB_URI config var from the writer app and setting it on the Node.js web app.

Note that the `MONGOLAB_URI` includes your connection username and password. Please keep it confidential

:::term
$ heroku config:add -a tp-web `heroku config -a tp-writer -s | grep MONGOLAB_URI`
Adding config vars and restarting app... done, v23
  MONGOLAB_URI => mongodb://heroku...eroku_app123456

Removing the `mongolab` add-on from `tp-writer`, or destroying the app itself, will irreversibly de-provision the database even though it's still referenced from `tp-web`. Be careful of such situations when working with shared resources.

Deploy web app

Applications deployed with a `web` process type will automatically be scaled to one web dyno.

Deploy the Node.js app to Heroku and check the status of the web process with heroku ps.

:::term
$ git push heroku master
Counting objects: 30, done.
...
-----> Heroku receiving push
-----> Node.js app detected
...
-----> Launching... done, v3
       http://tp-web.herokuapp.com deployed to Heroku

$ heroku ps
=== web: `node app.js`
web.1: up for 40s

Run heroku open to open the application in your browser to see the JSON form of each message type being pushed, in real-time, from the Ruby writer app to the Node.js web app and finally to your browser.

Consuming messages in Node.js

The readAndSend function in app.js of the Node.js web app is responsible for consuming the messages sent to the capped collection by the ruby data writer component.

:::javascript
function readAndSend(socket, collection) {
  collection.find({}, {'tailable': 1, 'sort': [['$natural', 1]]}, function(err, cursor) {
    cursor.intervalEach(300, function(err, item) {
      if(item != null) {
        socket.emit('all', item);
      }
    });
  });
  // ...
};

The call to collection.find returns a cursor that iterates over all documents in the messages collection. The 'tailable' option specifies for the cursor to wait for additional data if it's reached the end of the result-set, thus mimicking real-time message receiving behavior.

For demonstration of listening to multiple queues an additional collection.find call does the same for only complex message types.

:::javascript
collection.find({'messagetype':'complex'}, {'tailable': 1, 'sort': [['$natural', 1]]}, function(err, cursor) {
  cursor.intervalEach(900, function(err, item) {
    if(item != null) {
      socket.emit('complex', item);
    }
  });
});

Notice that each iteration of the cursor emits the message document using socket.emit.

:::javascript
socket.emit('all', item);
// and
socket.emit('complex', item);

This pushes the message document (a JSON object) from the server to any connected browser clients. The library that powers this client-push feature is called Socket.IO.

Pushing messages to the browser with Socket.IO

The Node.js web application serves up a single index.html page that uses Socket.IO to open a connection to the server and create a listener attached to message types ‘all’ and ‘complex’.

:::javascript
var socket = io.connect('/');
socket.on('all', function (data) { ... }
socket.on('complex', function (data) { … }

True bi-directional messaging with [WebSockets](http://en.wikipedia.org/wiki/WebSocket) is not yet available.

In reality, the client is polling the server for more data as the server-side Socket.IO configuration forces the connection to utilize XHR-polling with a 10-second timeout.

:::javascript
io.configure(function () { 
  io.set("transports", ["xhr-polling"]); 
  io.set("polling duration", 10);
});

Once the browser has shown all available messages it will stop. As new messages are inserted into the database, the browser will be pushed the new messages and resume.

Conclusion

The four technologies covered in this article are just one of many combinations that support a componentized real-time app. More fundamental is the role MongoDB's data-flexibility and Cedar's polyglot capabilities play in eschewing monolithic applications for a more modular system design.