Skip to content

Install Process on Ubuntu 12.04

Jump to bottom Edit New page
KeithUSF edited this page Mar 15, 2013 · 26 revisions

Installing OpenTreeMap stack on a fresh Ubuntu 12.04 box.

This is a work in progress.

Install Packages via apt package manager:

Update your packaging info

apt-get update

Apache and PostgreSQL

apt-get install apache2
apt-get install postgresql
apt-get install postgis

Required libs

apt-get install python-django
apt-get install libapache2-mod-wsgi
apt-get install python-psycopg2
apt-get install binutils
apt-get install libgeos-3.2.2      # 3.2.0 on 11.04 Natty
apt-get install libgeos-c1         # @FIX: c1 instead of cl
apt-get install libgdal1-1.7.0     # 1.6.0 on 11.04 Natty
apt-get install libproj0
apt-get install python-django-tagging
apt-get install python-imaging
apt-get install python-xlrd
apt-get install python-feedparser
apt-get install python-memcache
apt-get install python-beautifulsoup
apt-get install python-django-debug-toolbar
apt-get install python-simplejson
apt-get install python-django-extensions
apt-get install python-django-threadedcomments  # @FIX: add this
apt-get install python-gdal
apt-get install gdal-bin

Other necessities

apt-get install git
apt-get install subversion # Really only currently necessary to get django_reputation
apt-get install python-pip

Install a variety of other required Python libs

The python setup.py install lines below may need sudo.

mkdir ~/libs

cd ~/libs
wget http://django-template-utils.googlecode.com/files/template_utils-0.4p2.tar.gz
tar xzf template_utils-0.4p2.tar.gz
cd template_utils-0.4p2
python setup.py build
python setup.py install

cd ~/libs
wget -O django-profiles.tar.gz https://bitbucket.org/ubernostrum/django-profiles/get/tip.tar.gz
tar xzf django-profiles.tar.gz
cd ubernostrum-django-profiles-c21962558420
python setup.py build
python setup.py install

cd ~/libs
wget http://pypi.python.org/packages/source/U/Unidecode/Unidecode-0.04.5.tar.gz
tar xzf Unidecode-0.04.5.tar.gz
cd Unidecode-0.04.5
python setup.py build
python setup.py install

cd ~/libs
wget http://geopy.googlecode.com/files/geopy-0.94.tar.gz
tar xzf geopy-0.94.tar.gz
cd geopy-0.94
python setup.py build
python setup.py install

cd ~/libs
wget http://django-pagination.googlecode.com/files/django-pagination-1.0.5.tar.gz
tar xzf django-pagination-1.0.5.tar.gz
cd django-pagination-1.0.5
python setup.py build
python setup.py install

cd ~/libs
# @FIX: the URL listed gives a 404
wget http://pypi.python.org/packages/source/s/sorl-thumbnail/sorl-thumbnail-3.2.5.tar.gz
tar xzf sorl-thumbnail-3.2.5.tar.gz
cd sorl-thumbnail-3.2.5
python setup.py build
python setup.py install
# @FIX: updated to 11.12. Should we just install via pip in the first place?
pip install --upgrade sorl-thumbnail

cd ~/libs
wget http://sourceforge.net/projects/dbfpy/files/dbfpy/2.2.5/dbfpy-2.2.5.tar.gz
tar xzf dbfpy-2.2.5.tar.gz
cd dbfpy-2.2.5
python setup.py build
python setup.py install

cd ~/libs
wget -O django-badges.tar.gz https://bitbucket.org/jiaaro/django-badges/get/tip.tar.gz
tar xzf django-badges.tar.gz
cd jiaaro-django-badges-07526b7f7d6f
python setup.py build
python setup.py install

cd ~/libs
wget https://bitbucket.org/ubernostrum/django-registration/downloads/django-registration-0.8-alpha-1.tar.gz
tar xzf django-registration-0.8-alpha-1.tar.gz
cd django-registration-0.8-alpha-1
python setup.py build
python setup.py install

cd ~/libs
git clone git://github.com/miracle2k/webassets.git
cd webassets
python setup.py build
python setup.py install

pip install django-pipeline

# @FIX: installing south was not listed in the docs
pip install south

# OPTIONAL: Install python-omgeo the Oatmeal Geocoder
cd ~/libs
git clone https://github.com/azavea/python-omgeo.git
cd python-omgeo
python setup.py build
python setup.py install

Fix PROJ.4 to deal with Spherical Mercator

cd ~/libs
wget http://download.osgeo.org/proj/proj-datumgrid-1.4.tar.gz
mkdir proj-datumgrid-1.4
tar xzf proj-datumgrid-1.4.tar.gz -C proj-datumgrid-1.4
cd proj-datumgrid-1.4
nad2bin null < null.lla
sudo cp null /usr/share/proj

Install django-shapes, first removing HttpResponse call

cd ~/libs
wget -O django-shapes.tar.gz https://bitbucket.org/springmeyer/django-shapes/get/tip.tar.gz
tar xzf django-shapes.tar.gz
cd springmeyer-django-shapes-f3c93240c56e

In shapes/views/export.py in the zip_response() method, comment out lines 117-123 and instead, return zip_stream.

It should look like this:

        # Stick it all in a django HttpResponse
        #response = HttpResponse()
        #response['Content-Disposition'] = 'attachment; filename=%s.zip' % file_name.replace('.shp','')
        #response['Content-length'] = str(len(zip_stream))
        #response['Content-Type'] = mimetype
        #response.write(zip_stream)
        #return response

        return zip_stream

Then install.

python setup.py build
python setup.py install

Install django-reputation with a fix

cd ~/libs
svn checkout http://django-reputation.googlecode.com/svn/trunk/ django-reputation
cd django-reputation
sudo cp django_reputation -R /usr/local/lib/python2.7/dist-packages/django_reputation

Now we change default config and user because of a bug where it doesn't seem to accept values in settings.py):

cd /usr/local/lib/python2.7/dist-packages/django_reputation

In config.py, change values as needed. (?)

In models.py, make the following changes:

    ....
    relevant_reputation_actions = UserReputationAction.objects.filter(user=user).filter........
    ....
    if expected_delta <= MAX_REPUTATION_GAIN_PER_DAY and expected_delta >= -1 * MAX_REPUTATION_LOSS_PER_DAY:
        delta = action_value
    elif expected_delta > MAX_REPUTATION_GAIN_PER_DAY:
        delta = 0
    elif expected_delta < MAX_REPUTATION_LOSS_PER_DAY:
        delta = 0
    ...

Install django-sorting with patch

cd ~/libs
git clone git://github.com/directeur/django-sorting.git
cd django-sorting
# Apply changes from https://github.com/directeur/django-sorting/issues#issue/8
# Using patch in gist:
git clone git://gist.github.com/3470505.git
mv 3470505/django_sorting-autosort-2.patch .
rm -rf 3470505
patch -p0 < django_sorting-autosort-2.patch
cp django_sorting -R /usr/local/lib/python2.7/dist-packages/django_sorting

Get OpenTreeMap source

May need sudo.

cd /var/www
git clone --branch v1.2 https://github.com/azavea/OpenTreeMap.git opentreemap

Set up Database

Create Create PostGIS database template

(Mostly via https://gist.github.com/1481128) May need some sudos. Also, before running the ./ configure step, it may behoove you to run

sudo apt-get install postgresql-server-dev-9.1

(or whatever PG version we're up to when you read this). Beginner's note: If Ubuntu asks you for a postgres user password that you don't know, enter

sudo su postgres

first. That will get you logged in as the postgres user, and you can run all of the red commands below in single-quotes right in the cli.

apt-get install postgresql-contrib libpq-dev checkinstall proj libgeos-dev libxml2 libxml2-dev
sudo mkdir -p '/usr/share/postgresql/9.1/contrib/postgis-1.5'
cd /usr/share/postgresql/9.1/contrib/postgis-1.5
wget http://postgis.refractions.net/download/postgis-1.5.5.tar.gz
tar zxvf postgis-1.5.5.tar.gz && cd postgis-1.5.5/
./configure && sudo make && sudo checkinstall --pkgname postgis-1.5.5 --pkgversion 1.5.5-src --default
su postgres -c'createdb -E UTF8 -U postgres template_postgis'
su postgres -c'createlang -d template_postgis plpgsql;'
su postgres -c'psql -U postgres -d template_postgis -c "CREATE EXTENSION hstore;"'
su postgres -c'psql -U postgres -d template_postgis -f /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql'
su postgres -c'psql -U postgres -d template_postgis -f /usr/share/postgresql/9.1/contrib/postgis-1.5/spatial_ref_sys.sql'
su postgres -c'psql -U postgres -d template_postgis -c "select postgis_lib_version();"'
su postgres -c'psql -U postgres -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"'
su postgres -c'psql -U postgres -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"'
su postgres -c'psql -U postgres -d template_postgis -c "GRANT ALL ON geography_columns TO PUBLIC;"'

Set password [from postgres prompt]

sudo -u postgres psql postgres
\password postgres

CTRL-D to exit postgres prompt.

Create a database using the template.

sudo -u postgres createdb -T template_postgis YOURDBNAME

Import Choices.sql into the database.

(andrewbt note: this may be deprecated/unnecessary in v1.2, need to double check)

# Choices.sql is available inside opentreemap folder
sudo -u postgres psql YOURDBNAME -f /var/www/opentreemap/Choices.sql

Enable non-peer login with postgres.

We're going to have the app identify itself as the main 'postgres' user, which is pretty rudimentary and somewhat unsafe.

Edit /etc/postgresql/9.1/main/pg_hba.conf and change:

    local   all             postgres                                peer

to

    local   all             postgres                                md5

Turn off standard_conforming_strings

Default PostgreSQL 9.1 and/or psycopg2 installs/configs currently have issues with backslashes in string literals (causing an issue with OTM v1.2 json calls on the Add a New Tree page). The fix is to force that setting off.

Edit /etc/postgresql/9.1/main/postgresql.conf and specify:

standard_confirming_strings = off

Restart postgres

sudo service postgresql restart

Set up your settings_YOURLOCATION.py file.

Get copies of other cities' settings_LOCATION.py files -- like from https://github.com/jlivni/OpenTreeMap. I copied Oakland's as settings_MYLOCATION.py, and tweaked.

Add to settings.py, before 'OTM_VERSION = "1.2"' :

    from settings_YOURLOCATION import *

Change DATABASES definition with db credentials:

    DATABASES = {
        'default': {
            'NAME': 'YOURDBNAME',
            'ENGINE': 'django.contrib.gis.db.backends.postgis',
            'USER': 'postgres',
            'HOST': '',
            'PASSWORD': 'YOURDBPASSWORD',
            'PORT': '5432',
        }
    }

Set up your choices_LOCATION.py file.

cd /var/www/opentreemap
cp treemap/choices.py choices_LOCATION.py
# add to settings.py:
    from choices_LOCATION import *

Run install script

cd /var/www/opentreemap
python manage.py syncdb
python manage.py migrate

If you get errors

Put this in your local_settings.py, just to start:

ADD_FORM_TARGETS = [
('addsame', 'Add another tree using the same details'),
('add', 'Add another tree with new details'),
('edit','Continue editing this tree'),
('view', "I'm finished"),
]
ADD_FORM_TARGETS_DEFAULT = 'view'
API_KEY_GOOGLE_MAPS = ''
API_KEY_GOOGLE_ANALYTICS = ''

Get a template to start with, like this directory: https://github.com/jlivni/OpenTreeMap/tree/master/templates/Oakland

Filesystem/Apache Setup/Config

  • Add write permissions to the /static/ and /media/ directories (TODO: add best way to do this - change directory owner to "www-data"?)
  • Ensure Django admin media is available (e.g. specify "admin_media" as ADMIN_MEDIA_PREFIX in implementation settings file and then add a symbolic link to that directory: sudo ln -s /usr/lib/python2.7/dist-packages/django/contrib/admin/media admin_media)

Get and import Zip Code data

We need to populate the treemap_zipcode table with the local zipcodes and their boundaries. You can do this in any number of ways. A generalized way of populating those tables is shown here using the Census' Zip Code Tabulation Areas, which won't always be accurate. See this OTM Google Group message for more info.

  1. Go to http://www.census.gov/cgi-bin/geo/shapefiles2010/main.
  2. Choose "Zip Code Tabulation Areas" and submit.
  3. Choose your state from the 2010 drop-down.
  4. Download the file into ~/data/zipcodes.

Now unzip, (installing the unzip utility if necessary):

cd ~/data/zipcodes
sudo apt-get install unzip
unzip -d tl_2010_39_zcta510 tl_2010_39_zcta510.zip

Convert the shapefile to a psql import script, and import the data into a temporary table.

shp2pgsql -s 4326 tl_2010_39_zcta510 treemap_zipcode_tmp > zips.sql
sudo -u postgres psql opentree -f zips.sql

Copy the necessary columns into treemap_zipcode, then remove the temporary table.

sudo -u postgres psql opentree -c 'INSERT INTO treemap_zipcode (zip, geometry) SELECT zcta5ce10, the_geom FROM treemap_zipcode_tmp;'
sudo -u postgres psql opentree -c 'DROP TABLE treemap_zipcode_tmp;'
Add a custom footer
Add a custom sidebar
Clone this wiki locally