-
Notifications
You must be signed in to change notification settings - Fork 2
/
NOTES.txt
131 lines (101 loc) · 7.35 KB
/
NOTES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
APPLICATION DESCRIPTION
The linz-admin-boundaries-ui is the front end for the linz-admin-boundaries-uploader python script (the description of this is located in that module).
Originally written to allow user updating of Admin-Boundaries layers, those sourced externally, it now also provides user management capabilities that allow user management for both the AdminBoundaries application and AIMS.
Performing a supporting function for AIMS, AdminBoundaries was built around existing scripts/processes that updated external data sources.
Many of these processes were manual and in some cases involved users copying data from emailed spreadsheets.
To lend a small level of automation to these processes the AdminBoundaries script was written to both parse CSV data from an internal CIFS share and extract shapefile data from a remote FTP administered but StatsNZ.
StatsNZ have since adapted their output and now provide layers over Kx hosted WFS.
The AdminBoundaries (AB) application operates as a tool to allow user verification of external data before allowing it into the AIMS environment. External layers are accessed and imported into an preview schema in the main "linz_db" database where users can inspect the data before committing the data to the production system. The UI component of this application facilitates the importation, inspection and management of this process.
The AB-ui presents 3 user screens.
1. The Import page shows the row counts on the configured layers in both the production system and those that have been recently imported. Operations to reimport/delete or transfer (into production) are provided on this page.
2. A Config page presents a form replicating the configuration data used by the importer script. Here the user can adjust connection setting, import parameters and layer conversions
3. The User page was written in support of AIMS whose user management interface was never delivered and deferred to a future release. This interface enables management of three levels of user access control. PostgreSQL group memberships (for AIMS), user/password management including encryption for tomcat (by editing the tomcat-users.xml file) and User login within AIMS itself.
BUILD
Gradle is used as the build manager. Because some dependencies are not available in the maven repositories the easiest workaround I have found is to set up a small ivy repository. (Ivy itself doesn't have to be installed, just the directory structure and format need to be set up)
The dependencies currently supplied through Ivy are; psycopg2, pggdal and ptyprocess
For deployment we use the gradle plugin com.bmuschko.cargo. It connects to tomcat for WAR upload/download.
The gradle targets used to build and deploy AB are:
compileJava: Java source to class
copyscripts: Copy the python repo into the WEB-INF/scripts
war: Packaging as Web Archive
deploy: Push the war to Tomcat (remembering to undeploy first)
Only deploy/undeploy are generally needed as compilation etc are treated as dependencies of these tasks.
compileJava (disambiguates compile from compileTestJava and compileJava) and builds the Java source
copyscripts copies the python script source (linz_admin_boundaries_uploader) into a WEBAPP nested subdirectory
war archives the java classes and the python scripts into a deployable Web jAR.
undeploy deletes the existing .war (installation) from the server
deploy pushes the active .war to the server.
The parameters on where to deploy to are set using the grade.properties file (NB like the python config, the template file in the repo will need to be modified)
NB the required parameters are; deployment server, deployment method (http, https) user and pass
NOTES
1. Parallel Resource
1.1 Add new resource in context.xml
<Resource name="jdbc/linz/dab"
...
driverClassName="org.postgresql.Driver"
url="<jdbc-url>"
validationQuery="SELECT 1"
defaultAutoCommit="true"
/>
1.2 Point connector at new resource
datasource = (DataSource) new InitialContext().lookup("java:comp/env/jdbc/linz/dab");
2. Role Restriction
2.1 Create new role in tomcat-users.xml
<role description="Role granting access to Downloader for Admin Boundaries" rolename="DAB"/>
2.2 Assigned selected users to DAB role
<user password="<enc-pass>" roles="DAB[,other,roles]" username="<admin-user>"/>
2.3 Set DAB web.xml security-role and auth-constraint
<web-app ...>
<security-role>
<role-name>DAB</role-name>
</security-role>
<security-constraint>
<auth-constraint>
<role-name>DAB</role-name>
</auth-constraint>
</security-constraint>
3. HTTPS
3.0 Create keystore with self signed key
keytool -genkey -alias <keystore-alias> -keyalg RSA -keystore [path/to/keystore]
3.1 Uncomment 8443 connector in server.xml
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
keystoreFile="[path/to/keystore]" keystorePass="<keystore-pass>"/>
3.2 Set DAB web.xml to enforce https
<web-app ...>
<security-constraint>
<user-data-constraint>
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
3.3 Create firewall rule allowing 8443 (as root) on the webserver
ufw allow from any to any port 8443 proto tcp
3.4 Clients accept self-signed cert
4. Build tomcat restart functionality
4.1 Write a simple restart function in tomcat/bin
#!/bin/sh
/opt/tomcat8/bin/shutdown.sh
sleep 5
/opt/tomcat8/bin/startup.sh -security
> chmod 755 restart.sh
> chown tomcat8:tomcat8 restart.sh
4.2 Execute the restart from the DAB user webapp
Runtime.getRuntime().exec("/opt/tomcat8/bin/restart.sh");
5, Install py4j
5.1 scp a source copy of py4j to one of the geoservers
5.2 As root unzip and run python(3) setup.py install.
If this installs an egg! unpack using pip with the command
> pip(3) install .
in the source dir
5.3 Set permissions for the py4j files
> chmod 775 /usr/local/lib/python2.7/dist-packages/py4j/*
> chmod 775 /usr/local/lib/python2.7/dist-packages/py4j/tests/*
> chmod 775 /usr/local/lib/python3.4/dist-packages/py4j/tests/*
> chmod 775 /usr/local/lib/python3.4/dist-packages/py4j/*
NOTES (problems seen in the past and their fixes)
a. [UserAdmin won't load] Make sure the role set in the config has sufficient privileges to access the API and that TC has been restarted so this role, if new, is active
b. "Restart AA" will restart the DAB application. Use this if changes are not showing up or the application appears to have hung
c. "Restart TC" restarts Tomcat. For simple user changes this shouldn't cause any problems. Doing a restart with a new WAR is however not advised unless the WAR has been fully tested. Restarting by this method can sometimes cause tomcat to hang with a "org.apache.jasper.servlet.TldScanner.scanJars" error. If this is the case restart tomcat again manually.
d. [No TA bdys to import] TA boundaries are located on an internal file server and accessed through a CIFS mount /mnt/gisdata. The most common reason for TA bdys data to be unavaiable is the loss of this mount point. To fix this first attempt to remount and then check if the fileserver is available.
e. [New user cant login] Ensure when restarting tomcat that either the restart.sh script is called or the -security option is appended to the startup.sh call. This enables the SecurityManager which authenticates added users