Hive FDW for PostgreSQL

This Python module implements the multicorn.ForeignDataWrapper interface to allow you to create foreign tables in PostgreSQL 9.1+ that query to tables in Apache Hive.

Pre-requisites

• PostgreSQL 9.1+
• Python
• Multicorn
• hive-thrift-py

Installation

1. Install Multicorn

2. Install hive-thrift-py

3. Build the FDW module:

    $ cd hive-fdw-for-postgresql
    $ python setup.py sdist
    $ sudo python setup.py install
   

   or, with easy_install:

    $ cd hive-fdw-for-postgresql
    $ sudo easy_install .
   

4. In the PostgreSQL client, create an extension and foreign server:

    CREATE EXTENSION multicorn;
    
    CREATE SERVER multicorn_hive FOREIGN DATA WRAPPER multicorn
    OPTIONS (
        wrapper 'hivefdw.HiveForeignDataWrapper'
    );
   

Examples

1. User can executes simple selects on a remote Hive table:

    CREATE FOREIGN TABLE hive (
        a varchar,
        b varchar,
        c varchar,
        d varchar
    ) SERVER multicorn_hive OPTIONS (
        host 'tb081',
        port '10000',
        table 'test'
    );
   
    SELECT * FROM hive;
   

2. Also user can executes selects using a Hive query:

    CREATE FOREIGN TABLE hive_query (
        x varchar,
        y varchar,
        z varchar
    ) SERVER multicorn_hive OPTIONS (
        host 'tb081',
        port '10000',
        query 'SELECT x,y,z from src'
    );
    
    SELECT * from hive_query;