Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a Tables interface for features #118

Merged
merged 70 commits into from
Sep 14, 2020
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
a8bda52
Initial Commit(setup)
Sov-trotter Mar 29, 2020
08c3e55
Add Tables dependency
Sov-trotter Mar 30, 2020
e7b0edc
Setup GeoTable, reading and parsing files[broken]
Sov-trotter Mar 30, 2020
5fbda99
Create function for returning a NamedTuple
Sov-trotter Mar 31, 2020
fee1404
Remove extra dependency
Sov-trotter Apr 1, 2020
0448998
Implement Tables interface
Sov-trotter Apr 3, 2020
8f33687
add struct GeoTable
Sov-trotter Apr 6, 2020
59a3f29
work towards a row based Tables interface
visr Apr 12, 2020
fb98c7b
Reset local fork and local repo made changes
Sov-trotter Apr 20, 2020
3746305
Import Tables instead of DataStreams
Sov-trotter Apr 20, 2020
31cad0b
Add Base.iterate test
Sov-trotter Apr 21, 2020
67b4336
Modify tests
Sov-trotter Apr 21, 2020
3f937a1
Merge branch 'tables' into dev-tables
Sov-trotter Aug 10, 2020
82ee8a0
Resolve conflict
Sov-trotter Aug 10, 2020
1484024
Merge pull request #2 from yeesian/master
Sov-trotter Aug 10, 2020
86f588b
Minor Refeactor
Sov-trotter Aug 10, 2020
4f5d9c5
Clean up
Sov-trotter Aug 10, 2020
e6b0e9e
Update test
Sov-trotter Aug 10, 2020
23cc9f1
Schema test for 1.5
Sov-trotter Aug 10, 2020
cf68ff1
Make schema test more concrete
Sov-trotter Aug 11, 2020
78fc8fa
Refactor test dependencies
Sov-trotter Aug 11, 2020
5c35700
Tables dependency. :(
Sov-trotter Aug 11, 2020
3101904
Improve iterate method
Sov-trotter Aug 11, 2020
649c941
Work out extra methods; clean up stuff
Sov-trotter Aug 11, 2020
0c9f982
Pass through the geotable function first
Sov-trotter Aug 11, 2020
ade232f
Minor Refactor
Sov-trotter Aug 12, 2020
1a0537c
Take Geometry Out of the table
Sov-trotter Aug 12, 2020
7774256
(no branch):
Sov-trotter Aug 15, 2020
0730dce
Accept null geometries
Sov-trotter Aug 15, 2020
4888ffe
Merge branch 'master' into dev-tables
Sov-trotter Aug 15, 2020
4bf1f0d
Support mutiple geomtries; rename to geotable to Table
Sov-trotter Aug 16, 2020
385d80e
Add tests
Sov-trotter Aug 16, 2020
edfd3e7
Merge pull request #4 from yeesian/master
Sov-trotter Aug 17, 2020
45196aa
Add docs for tables
Sov-trotter Aug 17, 2020
86a2882
CSV doc
Sov-trotter Aug 17, 2020
3ccdf96
Add sha details
Sov-trotter Aug 17, 2020
7f11a47
Merge pull request #5 from yeesian/master
Sov-trotter Aug 17, 2020
5b297d2
Merge pull request #6 from yeesian/master
Sov-trotter Aug 18, 2020
81975ee
Merge pull request #7 from Sov-trotter/dev-tables
Sov-trotter Aug 18, 2020
9b16b07
Merge branch 'docs-tables' into dev-tables
Sov-trotter Aug 18, 2020
fbdef49
Pass through the Table function
Sov-trotter Aug 18, 2020
c1b7ac4
nit's
Sov-trotter Aug 19, 2020
e9949d4
Refactor Base.iterate
Sov-trotter Aug 22, 2020
91f6489
Remove AG; refactor methods
Sov-trotter Aug 22, 2020
b9d0d6b
inline ngeom
Sov-trotter Aug 22, 2020
a4bf268
Minor refactor
Sov-trotter Aug 23, 2020
803691b
Typos fix
Sov-trotter Aug 23, 2020
1655a18
remove DataStreams from deps
visr Aug 23, 2020
a0450f7
wrap long lines in docs
visr Aug 23, 2020
2156aec
using instead of import, to force explicit override
visr Aug 23, 2020
2aa7499
extend getlayer for Table
visr Aug 23, 2020
c196b3a
support passing field name as symbol
visr Aug 23, 2020
d933f09
allow only creating tables from layers
visr Aug 23, 2020
e84491d
various other fixups
visr Aug 23, 2020
90e148f
Refactor schema to include geometry names/types
Sov-trotter Aug 23, 2020
924df47
Add getindex method
Sov-trotter Aug 23, 2020
cee6cec
Update docs
Sov-trotter Aug 23, 2020
4bd8c5a
Add/update tests
Sov-trotter Aug 23, 2020
bcec31e
Fix scope in tests
Sov-trotter Aug 23, 2020
d245b12
Clean up methods
Sov-trotter Aug 23, 2020
513fe65
Refaactors; pretty code :)
Sov-trotter Aug 23, 2020
fadaa78
nit's
Sov-trotter Aug 26, 2020
07f210c
Fix path in tests
Sov-trotter Aug 26, 2020
eff39c9
Add nextnamedtuple to docs
Sov-trotter Aug 26, 2020
7677ae7
typo fix
Sov-trotter Aug 27, 2020
87eb4f7
Test for null/missing
Sov-trotter Aug 31, 2020
d5c0ed3
Refactor tests
Sov-trotter Sep 1, 2020
be2df2a
nit's
Sov-trotter Sep 1, 2020
3247789
1-based indexing
Sov-trotter Sep 13, 2020
84380a5
32bit fix
Sov-trotter Sep 14, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ desc = "A high level API for GDAL - Geospatial Data Abstraction Library"
version = "0.3.2"

[deps]
DataStreams = "9a8bc11e-79be-5b39-94d7-1ccc349a1a85"
Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
GDAL = "add2ef01-049f-52c4-9ee2-e494f65e021a"
GeoFormatTypes = "68eda718-8dee-11e9-39e7-89f7f65f511f"
GeoInterface = "cf35fbd7-0cd7-5166-be24-54bfbe79505f"

[compat]
DataStreams = "0.4.2"
Tables = "1"
GDAL = "1.0.2"
GeoInterface = "0.4, 0.5"
GeoFormatTypes = "0.3"
Expand Down
4 changes: 2 additions & 2 deletions src/ArchGDAL.jl
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
module ArchGDAL

import GDAL, GeoInterface, GeoFormatTypes
import DataStreams: Data
import Tables: Tables
import GeoInterface: coordinates, geotype
import Base: convert

Expand Down Expand Up @@ -30,7 +30,7 @@ module ArchGDAL
include("context.jl")
include("base/iterators.jl")
include("base/display.jl")
include("datastreams.jl")
include("tables.jl")
include("geointerface.jl")
include("convert.jl")

Expand Down
55 changes: 0 additions & 55 deletions src/datastreams.jl

This file was deleted.

108 changes: 108 additions & 0 deletions src/tables.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
const AG = ArchGDAL
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved

struct GeoTable{T<:Union{IFeatureLayer, FeatureLayer}}
layer::T
end

function GeoTable(layer)
featuredefn = layerdefn(layer)
ngeometries = ngeom(featuredefn)

if(ngeometries == 0 )
error("Layer is not a valid ArchGDAL layer")
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
end
GeoTable(layer)
end

struct FeatureRow
Row::Array
feature_number::Int
end

function Tables.schema(layer::AG.AbstractFeatureLayer)
nfield = AG.nfield(layer)
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
featuredefn = AG.layerdefn(layer)
ngeom = ArchGDAL.ngeom(featuredefn)

fielddefns = (AG.getfielddefn(featuredefn, i) for i in 0:nfield-1)
geomdefns = (ArchGDAL.getgeomdefn(featuredefn, i) for i in 0:ngeom-1)

names_fields = Tuple(AG.getname(fielddefn) for fielddefn in fielddefns)
geom_names = Tuple(ArchGDAL.getname(geomdefn) for geomdefn in geomdefns)
names = (names_fields..., geom_names...)

types_fields = Tuple(AG._FIELDTYPE[AG.gettype(fielddefn)] for fielddefn in fielddefns)
geom_types = Tuple(ArchGDAL.gettype(geomdefn) for geomdefn in geomdefns)
types = (types_fields..., geom_types...)

Tables.Schema(names, types)
end

function Base.iterate(gt::GeoTable, st = 0)
layer = gt.layer
AG.resetreading!(layer)
nfeat = AG.nfeature(layer)
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
nfield = AG.nfield(layer)
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
ngeom = AG.ngeom(layer)
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
featuredefn = layerdefn(layer)

d = Dict{String, Vector}()

for field_no in 0:nfield-1
field = getfielddefn(featuredefn, field_no)
name = getname(field)
typ = _FIELDTYPE[gettype(field)]
d[name] = typ[]
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
end
d["geometry"] = IGeometry[]

st >= nfeat && return nothing
AG.nextfeature(layer) do feature
for (k, v) in pairs(d)
if k == "geometry"
val = getgeom(feature, 0)
else
val = getfield(feature, k)
end
push!(v, val)
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
end
end

keys_tup = ()
for _key in keys(d)
keys_tup = (keys_tup..., Symbol(_key))
end
vals_tup = Tuple(values(d))

#Using the tables interface
Row = Tables.rowtable(NamedTuple{keys_tup}(vals_tup))
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved

return FeatureRow(Row, st), st + 1
end

Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved

Tables.istable(::Type{<:GeoTable}) = true
Tables.rowaccess(::Type{<:GeoTable}) = true
Tables.rows(gt::GeoTable) = gt

Base.IteratorSize(::Type{<:GeoTable}) = Base.HasLength()
Base.size(gt::GeoTable) = nfeature(gt.layer)
Base.length(gt::GeoTable) = (Base.size(gt::GeoTable)) * (nfield(gt.layer) + ngeom(gt.layer))
Base.IteratorEltype(::Type{<:GeoTable}) = Base.HasEltype()

Base.propertynames(fr::FeatureRow, gt::GeoTable) = (Tables.schema(gt.layer)).names
#geometry
# geometry(fr::FeatureRow) = (fr.Row)
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved

Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved

function Base.show(io::IO, gt::GeoTable)
println(io, "GeoTable with $(ArchGDAL.nfeature(gt.layer)) Features")
end
# function Base.show(io::IO, fr::FeatureRow)
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
# println(io, "Feature with properties $(propertynames(fr,gt))")
# end
Base.show(io::IO, ::MIME"text/plain", gt::GeoTable) = show(io, gt)
Base.show(io::IO, ::MIME"text/plain", fr::FeatureRow) = show(io, fr)


Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved

Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion test/runtests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ include("remotefiles.jl")
cd(dirname(@__FILE__)) do
isdir("tmp") || mkpath("tmp")
include("test_convert.jl")
include("test_datastreams.jl")
include("test_tables.jl")
include("test_gdal_tutorials.jl")
include("test_geometry.jl")
include("test_types.jl")
Expand Down
19 changes: 0 additions & 19 deletions test/test_datastreams.jl

This file was deleted.

34 changes: 34 additions & 0 deletions test/test_tables.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
using Test
import ArchGDAL; const AG = ArchGDAL
using Tables

@testset "Tables Support" begin
# dataset = AG.read("point.geojson")
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
dataset = AG.read(joinpath(@__DIR__, "data/point.geojson"))
layer = AG.getlayer(dataset, 0)

nfeat = AG.nfeature(layer)
nfield = AG.nfield(layer)
featuredefn = AG.layerdefn(layer)
ngeometries = AG.ngeom(featuredefn)

#testing the main GeoTable function
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
@test sprint(print, AG.GeoTable(layer)) == "GeoTable with 4 Features\n"
@test sprint(print, AG.GeoTable(layer)) != "Layer is not a valid ArchGDAL layer\n"

# #testing Tables.schema
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
# @test sprint(print, AG.Tables.schema(layer)) == "Tables.Schema:\n :FID Float64 \n :pointname String \n Symbol(\"\") GDAL.wkbPoint"
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
# #testing Base.iterate
# @test sprint(print, AG.Base.iterate(AG.GeoTable(layer))) == "(ArchGDAL.FeatureRow(NamedTuple{(:pointname, :geometry, :FID),Tuple{String,ArchGDAL.IGeometry,Float64}}[(pointname = \"point-a\", geometry = Geometry: POINT (100 0), FID = 2.0)], 0), 1)"
Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved

@test Tables.istable(AG.GeoTable(layer)) == true
@test Tables.rowaccess(AG.GeoTable(layer)) == true
@test Tables.rows(AG.GeoTable(layer)) == AG.GeoTable(layer)

@test Base.size(AG.GeoTable(layer)) == 4
@test Base.length(AG.GeoTable(layer)) == 12

Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved
end



Sov-trotter marked this conversation as resolved.
Show resolved Hide resolved