Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC Osm pylint #62

Closed
wants to merge 43 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
add9f2d
http://download.iiab.io/content/OSM/vector-tiles/maplist/hidden/regio…
jvonau Apr 26, 2022
4d61e33
Lets keep the new file in a new directory
jvonau Apr 27, 2022
1185067
Lets keep the new file in a new directory - 2020
jvonau Apr 28, 2022
306ae9c
Merge pull request #54 from jvonau/github_host
holta Apr 28, 2022
22fa3df
Refine iiab-install-map-region for *2020*.mbtiles & Contabo
holta Apr 30, 2022
1c203bc
Merge pull request #58 from holta/update_iiab-install-map-region
holta Apr 30, 2022
73e3896
Updated & Friendlier docs/README.md
holta May 1, 2022
18777af
docs/README.md: Links/context + punctuation
holta May 1, 2022
ae6640f
docs/README.md: Resilient link to FAQ item 35
holta May 1, 2022
35285da
docs/README.md: Tighten up intro + typography + Future
holta May 1, 2022
fe09218
docs/README.md: Clarify satellite photo pixels represent 19 x 19m on …
holta May 1, 2022
8bc4cf8
docs/README.md: Clarify continent/region install instructions
holta May 1, 2022
96a6ee5
docs/README.md: Link to github.com/iiab/iiab-admin-console/search?q=osm
holta May 1, 2022
53d7ba0
docs/README.md: Update title + grammar/punct + iiab-factory (historical)
holta May 1, 2022
0528a2e
docs/README.md: OSM continent/region install options
holta May 1, 2022
7b2a73f
docs/README.md: De-emphasize iiab-factory
holta May 1, 2022
b012d97
docs/README.md: Other repos' README's = duplication but also context
holta May 1, 2022
b04e9dc
docs/README.md: Capitalization touch-up
holta May 1, 2022
25b7201
docs/README.md: Link to Jan 2021 "Recipe for Generating Map Regions"
holta May 1, 2022
f14f956
docs/README.md: Clarify link to roles/osm-vector-maps README ("What's…
holta May 1, 2022
b72a712
docs/README.md: Dev notes + sat photo install options
holta May 2, 2022
18b07f2
docs/README.md: Attribute source data to MapTiler (both URL's) & Sent…
holta May 2, 2022
7306a1c
Merge pull request #59 from holta/readme-polish
holta May 2, 2022
f0a9574
docs/README.md: Link to cities1000.sqlite iiab/iiab#3206 for PR #59
holta May 2, 2022
20e8f64
docs/README.md: Link to map catalog(s) for PR #59
holta May 2, 2022
4220953
docs/README.md: Clarify "Log in to IIAB's Admin Console" for #59
holta May 2, 2022
b999a2b
iiab-maps-finish.py whitespace
jvonau May 3, 2022
9c6f357
iiab-make-init.py whitespace
jvonau May 3, 2022
0cd0758
iiab-make-init.py indents
jvonau May 3, 2022
51a704c
iiab-make-init.py - full pylint
jvonau May 3, 2022
4e942a9
indents
jvonau May 3, 2022
be34d3b
constants
jvonau May 3, 2022
47f6090
imports
jvonau May 3, 2022
ab9c0cb
map is a python reserved word
jvonau May 3, 2022
72d8882
unused
jvonau May 3, 2022
a72afd6
docstrings
jvonau May 3, 2022
0dbefab
pylint C0321
jvonau May 3, 2022
acc5059
namespaces
jvonau May 3, 2022
667cc6c
make function
jvonau May 3, 2022
324eb03
pep8
jvonau May 3, 2022
54784a6
unused
jvonau May 4, 2022
324a7f2
get -> source_catalog
jvonau May 4, 2022
44869d5
Merge branch 'master' into osm_pylint
jvonau Jul 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added 2020/cities1000.sqlite
Binary file not shown.
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
2. The world's landmasses are covered by `detail.mbtiles -> <regional selection of OSM data>.mbtiles` (typically 2-30 GB, depending on region) at zoom levels 11-14 and overzoomable to level 18 — encoded as MVT/PBF vector maps.
3. Satellite imagery of the World is covered by `satellite.mbtiles -> satellite_z0-z9_2020.mbtiles` (1.2 GB) at zoom levels 0-9, encoded as JPEG bitmap/raster imagery. (IIAB also allows you to add to this file, supplementing it with satellite photos for specific regions you care about most.)


#### 5 Important Repos

- [github.com/iiab/maps](https://github.com/iiab/maps) (right here!)
Expand Down
52 changes: 20 additions & 32 deletions osm-source/pages/viewer/scripts/iiab-make-init.py
Original file line number Diff line number Diff line change
@@ -1,66 +1,54 @@
#!/usr/bin/python3
#!/usr/bin/python3
# -*- coding: UTF-8 -*-

import iiab.iiab_lib as iiab


"""Writes init.json """
import sys
import argparse
import sys, os
import json
import glob
import shutil
import json

# GLOBALS
viewer_path = '/library/www/osm-vector-maps/viewer'
catalog_path = '/etc/iiab'
map_catalog = {}
VIEWER_PATH = '/library/www/osm-vector-maps/viewer'
CATALOG_PATH = '/etc/iiab/map-catalog.json'

if len(sys.argv) != 2:
print("Argument 1=map_url")
sys.exit(1)
print("Argument 1=map_url")
sys.exit(1)

def get_map_catalog():
global map_catalog
input_json = '/etc/iiab/map-catalog.json'
"""returns CATALOG_PATH entries"""
input_json = CATALOG_PATH
with open(input_json, 'r') as regions:
reg_str = regions.read()
map_catalog = json.loads(reg_str)
#print(json.dumps(map_catalog, indent=2))
return map_catalog

def subproc_cmd(cmdstr, shell=False):
args = shlex.split(cmdstr)
outp = subproc_check_output(args, shell=shell)
return (outp)

def parse_args():
"""returns parsed args"""
parser = argparse.ArgumentParser(description="Create init.json for a tile URL.")
parser.add_argument("map_url", help="The 'detail_url' field in mapcatalog.json.")
return parser.parse_args()

def main():
global map_catalog
"""create init.json which sets initial coords and zoom"""
args = parse_args()
map_catalog = get_map_catalog()
catalog = map_catalog['maps']
#for k in catalog.keys():
#print(k)
map = catalog.get(args.map_url,{})
if len(map) == 0:
map2 = catalog.get(args.map_url,{})
if len(map2) == 0:
print('Download URL not found in map-catalog.json: %s'%args.map_url)
sys.exit(1)

# create init.json which sets initial coords and zoom
init = {}
map = catalog[args.map_url]
init['region'] = map['region']
init['zoom'] = map['zoom']
init['center_lon'] = map['center_lon']
init['center_lat'] = map['center_lat']
init_fn = viewer_path + '/init.json'
map3 = catalog[args.map_url]
init['region'] = map3['region']
init['zoom'] = map3['zoom']
init['center_lon'] = map3['center_lon']
init['center_lat'] = map3['center_lat']
init_fn = VIEWER_PATH + '/init.json'
with open(init_fn,'w') as init_fp:
init_fp.write(json.dumps(init,indent=2))

if __name__ == '__main__':
main()
main()
111 changes: 61 additions & 50 deletions osm-source/pages/viewer/scripts/iiab-maps-finish.py
Original file line number Diff line number Diff line change
@@ -1,47 +1,58 @@
#!/usr/bin/python3
#!/usr/bin/python3
# -*- coding: UTF-8 -*-

import iiab.iiab_lib as iiab


"""Create the idx file in format required by js-menu system"""
import sys
import os
import argparse
import sys, os
import json
import glob
import shutil
import json

# GLOBALS
viewer_path = '/library/www/osm-vector-maps/viewer'
vector_map_idx_dir = '/library/www/html/common/assets'
catalog_path = '/etc/iiab'
map_catalog = {}
VIEWER_PATH = '/library/www/osm-vector-maps/viewer'
VECTOR_MAP_IDX_PATH = '/library/www/html/common/assets/vector-map-idx.json'
CATALOG_PATH = '/etc/iiab/map-catalog.json'

if len(sys.argv) != 2:
print("Argument 1=map_url")
sys.exit(1)
print("Argument 1=map_url")
sys.exit(1)


def get_map_catalog():
global map_catalog
input_json = '/etc/iiab/map-catalog.json'
"""returns contents of CATALOG_PATH"""
input_json = CATALOG_PATH
with open(input_json, 'r') as regions:
reg_str = regions.read()
map_catalog = json.loads(reg_str)
#print(json.dumps(map_catalog, indent=2))
# print(json.dumps(map_catalog, indent=2))
return map_catalog

def subproc_cmd(cmdstr, shell=False):
args = shlex.split(cmdstr)
outp = subproc_check_output(args, shell=shell)
return (outp)

def write_init_json():
"""create init.json which sets initial coords and zoom"""
init = {}
args = parse_args()
source_catalog = get_map_catalog()
catalog = source_catalog['maps']
map3 = catalog[args.map_url]
init['region'] = map3['region']
init['zoom'] = map3['zoom']
init['center_lon'] = map3['center_lon']
init['center_lat'] = map3['center_lat']
init_fn = VIEWER_PATH + '/init.json'
with open(init_fn, 'w') as init_fp:
init_fp.write(json.dumps(init, indent=2))


def write_vector_map_idx(installed_maps):
# copied from adm_lib
"""copied from adm_lib"""
map_dict = {}
idx_dict = {}
map_catalog = get_map_catalog()

for fname in installed_maps:
map_dict = map_catalog['maps'].get(fname, '')
if map_dict == '': continue
if map_dict == '':
continue

# Create the idx file in format required bo js-menu system
item = map_dict['perma_ref']
Expand All @@ -53,50 +64,50 @@ def write_vector_map_idx(installed_maps):
idx_dict[item]['region'] = map_dict['region']
idx_dict[item]['language'] = map_dict['perma_ref'][:2]

with open(vector_map_idx_dir + '/vector-map-idx.json', 'w') as idx:
with open(VECTOR_MAP_IDX_PATH, 'w') as idx:
idx.write(json.dumps(idx_dict, indent=2))


def get_installed_tiles():
"""returns installed maps"""
installed_maps = []
tile_list = glob.glob(viewer_path + '/tiles/*')
tile_list = glob.glob(VIEWER_PATH + '/tiles/*')
for index in range(len(tile_list)):
if tile_list[index].startswith('sat'): continue
if tile_list[index].startswith('osm-planet_z0'): continue
installed_maps.append(os.path.basename(tile_list[index]))
if tile_list[index].startswith('sat'):
continue
if tile_list[index].startswith('osm-planet_z0'):
continue
installed_maps.append(os.path.basename(tile_list[index]))
return installed_maps


def parse_args():
parser = argparse.ArgumentParser(description="Create init.json for a tile URL.")
parser.add_argument("map_url", help="The 'detail_url' field in mapcatalog.json.")
"""returns parse args"""
parser = argparse.ArgumentParser(description="Create init.json for a tile \
URL.")
parser.add_argument("map_url", help="The 'detail_url' field in \
mapcatalog.json.")
return parser.parse_args()


def main():
global map_catalog
"""Create the idx file in format required by js-menu system"""
args = parse_args()
map_catalog = get_map_catalog()
catalog = map_catalog['maps']
#for k in catalog.keys():
#print(k)
map = catalog.get(args.map_url,{})
if len(map) == 0:
print('Download URL not found in map-catalog.json: %s'%args.map_url)
source_catalog = get_map_catalog()
catalog = source_catalog['maps']
# for k in catalog.keys():
# print(k)
map2 = catalog.get(args.map_url, {})
if len(map2) == 0:
print('Download URL not found in map-catalog.json: %s' % args.map_url)
sys.exit(1)

# create init.json which sets initial coords and zoom
init = {}
map = catalog[args.map_url]
init['region'] = map['region']
init['zoom'] = map['zoom']
init['center_lon'] = map['center_lon']
init['center_lat'] = map['center_lat']
init_fn = viewer_path + '/init.json'
with open(init_fn,'w') as init_fp:
init_fp.write(json.dumps(init,indent=2))

write_init_json()
installed_maps = get_installed_tiles()
print('installed_maps')
print(repr(installed_maps))
write_vector_map_idx(installed_maps)


if __name__ == '__main__':
main()
main()