Play Framework – Elastic Search Module

by Felipe Oliveira
http://mashup.fm
http://geeks.aretotally.in

Screencast
http://geeks.aretotally.in/play-elastic-search-module-new-0-0-5-release-with-screencast

Demo Source Code
https://github.com/mashup-fm/playframework-elasticsearch-0.0.5-demo

Searching is hard, just ask Google! Thankfully Lucene came about to provide incredible searching capabilities to the Java community. I have personally been using Lucene since 1999 and I am a huge fan, probably my very favorite Java project ever.

Lucene provides some great functionality but you still have a lot of work to do to integrate your app into Lucene; you need to map your models into a Lucene document, you need to write an indexer, you need to manage your indexes, cluster them, keep them in sync, write the search interface, etc. That’s a lot of work! To solve this problem there was a generation of frameworks built on top of Lucene, the most popular ones were Solr and Compass.

Compass and Solr are great frameworks, I have used them in production for very large deployments. I have built Foreclosure.com and Fannie Mae’s HomePath.com, Zillow.com’s Foreclosure Channel and an internal application to support HUD’s broker network for Florida, New York, New Jersey and Georgia entirely with that technology. There are still problems, it’s hard to cluster it, maintain real-time updates, backups, not have a single point of failure meaning failover which brings us to the next generation.

We finally come to Elastic Search from the awesome guys responsible for Compass! Elastic Search is the next generation of Compass, it’s a really great searching framework addressing a lot of the things that are so hard to deal with. It’s distributed, it supports Geo Spatial queries such bounding box searches, radius searches (around me type of search), backups on a slower data storages such as EC2, schema mapping, REST/JSON interface; like I said it’s really great stuff!

Elastic Search is a Distributed Search Solution based on Apache Lucene.

The main advantages are:

• Schema-Free (NoSQL)
• REST API Support – Easy Integration
• Reliable/No Single Point of Failure (you won’t lose any data or have any outage if a node goes broke, huge point when considered the next point also)
• Near Real-Time (pretty self explanatory)
• Async Write Behind for Long Term Storage (store data on a separate data center for example without affecting the performance of your application)
• Geospatial Support (forget about highly complex packages such as PostGIS)

Integrate Elastic Search in a Play! Framework Application. This module uses JPA events to notify Elastic Search of events of their own. In Local Mode it embeds a running Elastic Search instance (port 9200 by default), a good choice for development. In Client Mode it connects to an external instance of Elastic Search, that might be your setup in production.

Prerequisites

Play! 1.1

Install the module

Install the elasticsearch module from the modules repository:

play install elasticsearch

Install the module

Install the elasticsearch module from the modules repository:

play install elasticsearch

Enable the module

After installing the module, add the following to your conf/dependencies.yml to enable it (don’t forget to run play dependencies):

require:
	- play -> elasticsearch 0.5

Configure the module

You need to configure the module by setting these properties in your application.conf. There are two ways to run your Play! app with Elastic Search, local mode or client mode. Local mode works well for development purposes, an Elastic Search instance will run on the same JVM as your Play! application automatically. You won’t need to setup another service, etc. The second option is client mode which fits better in a production environment. The default option is local mode.

# Option 1) Elastic Search (Local Model)
elasticsearch.local=true

# Option 2) Elastic Search (Client Model)
elasticsearch.local=false
elasticsearch.client=mynode1:9200,mynode2:9200

Usage

You basically need to add annotation “@ElasticSearchable” to your Model class. It only works for JPA so far. If a model has that annotation our module we’ll watching for JPA events for instances of that class and route messages to Elastic Search to make sure Elastic Search index has the latest information available on the database.

Example:

@ElasticSearchable
@Entity
public class Post extends Model {
 
    public String title;
    public Date postedAt;
    
}

Searching

Simple Searching

SearchResults<Post> list = ElasticSearch.search(QueryBuilders.fieldQuery("title", "what a search"), Post.class);
See "Elastic Search documentation":http://www.elasticsearch.org/guide/reference/java-api/search.html for more example.

ElasticSearchController

The biggest change on this release is a nicer search interface. We are providing a very simple way to get started, inspired by Play!‘s CRUD module.
Basically you need to create a controller class and extend ElasticSearchController. Use annotation @ElasticSearchController.For(YOURMODELCLASS.class) to tell our module what model you want to search on, here’s an example:

@ElasticSearchController.For(ElasticSearchSampleModel.class)
public class ElasticSearchExample extends ElasticSearchController {

}

You should be able to search on http://localhost:9000/elasticSearchExample/index. If you want to customize the views, just create a directory ELASTIC_SEARCH under views and change whatever you need to change.

Indexing

The ElasticSearch Plugin features some strategies to configure the way your models are indexed.
These strategies implement the Interface play.modules.elasticsearch.IndexEventHandler. There are two built-in IndexEventHandler implementations and since 0.5 a way to add your own:

play.modules.elasticsearch.LocalIndexEventHandler

This IndexEventHandler uses a EventStream introduces in Play 1.2. to handle index events.
As the EventStream is limited to about 100 messages and then overflows, this is only a good candidate if you’re not importing lots of documents at a time.
The LocalIndexEventHandler is used by default or when setting

elasticsearch.delivery = LOCAL

in your application.conf

play.modules.elasticsearch.rabbitmq.RabbitMQIndexEventHandler

A IndexEventHandler using Akka and RabbitMQ for handling IndexEvents.
Use it by setting

elasticsearch.delivery = RABBITMQ

in your application.conf

Your own IndexEventHandler (new in 0.5)

Since 0.5 you can specify your own IndexEventHandler without modifying the Plugin. To do this, create a class implementing play.modules.elasticsearch.IndexEventHandler
somewhere in your Play Project, e.g. in a package called handlers and specify the following in your application.conf

elasticsearch.delivery = CUSTOM
elasticsearch.customIndexEventHandler = helpers.MyCustomIndexEventHandler

Make sure your custom IndexEventHandler implements the interface mentioned above and does have a default no-args constructor.

User Interface

After you start your application (play run), you should have an admin interface automatically running on http://localhost:9000/es-admin/.

Source Code

Fork it on Github https://github.com/feliperazeek/playframework-elasticsearch.

Roadmap

• Add different methods of notification (ElasticSearch River, AMQP, JMS)
• Adding support for non-Jpa models.
• Parent/Child support
• Customizations for mapping, etc
• OAuth Support

Credits

• Shay Banon for the great work with Elastic Search and Compass, I have been following his work for a few years, great stuff.
• Ben Birch (https://github.com/mobz) for the work on the User Interface.

Changelog

Version 0.0.9

• Embedded property support through @ElasticSearchEmbedded
• Fixed a bug which prevented model create- and update-events from being processed
• ElasticSearchController now uses elastic search for searching

Version 0.0.8

• JPA entity hydration support
• Query support