Skip to content

GNIP 14 New Search Backend

jj0hns0n edited this page Sep 29, 2012 · 4 revisions

GNIP 14 - New Search Backend

Overview

It is proposed to add a 'new' (additional) search back-end to GeoNode that can make use of a full text search engine which is capable of indexing all of the model classes in the system including Layers, Maps, Profiles and new classes such as Groups, Collections etc. This new search back-end should provide an API that can be used both by the html/js based search page as well as in by a GXP widget in the Map Composer (and by external applications).

Initial work under this proposal has been doing by Ian Schneider.

https://github.com/jj0hns0n/geonode/tree/new_search

Proposed By

Jeffrey Johnson

Assigned to Release

TBD

State

For Review and Comment

Motivation

Currently GeoNode's search is based almost entirely on making GetRecords queries to the CSW backend (GeoNetwork). Map search is accomplished by making database queries directly (although there does appear to be a GeoNetwork based Maps search). There is currently no way to search for profiles.

Several GeoNode users have complained about the quality of the search results from GeoNetwork.

There is also no way to filter search results by data type (raster, vector) or by date range etc. It may be possible to query the GeoNetwork catalog this way by using lucene query syntax, but this is not document/tested or used widely, and its not clear if it able to produce the results desired.

Proposal

It is proposed to implement Django Haystack http://haystacksearch.org/ as a full text search engine in GeoNodePy and connect the Layer, Map and Contact (Profile) classes to it. Additionally any new classes add should be able to be configured to be indexed by haystack in order to be searched. A new set of search views (methods) should be developed that make use of haystack for full text search and also provide for filtering by various criteria as detailed in wireframes here (rollie, where are these??). The developed views should constitute a documented set of API endpoints (primarily returning json) that allow the html search pages to make use of the search methods in order to present an easy to use search front end for the user. These API endpoints will also be used by a GXP widget in the map composer and potentially by external applications.

Django Haystack allows for various search back-ends to be configured (in much the same way that Django's own Database system works). The default backend is called dummy which simply uses database queries to do search. The Dummy backend will use whichever database is configured. Documentation should be provided on how to configure haystack with solr or other more powerful backends, but these should be optional.

Issues

TODO

Testing

A thorough set of integration tests should be developed to carefully test this new functionality in the context of a GeoNode site.

Alternatives

Do nothing. GeoNode's current search functionality works as is, but has severe limitations.

Feedback

See discussion on the geonode-dev list here.

https://groups.google.com/a/opengeo.org/group/geonode-dev/browse_thread/thread/a3c2a00f4ccfbd6a

Voting

Clone this wiki locally