FlatGeobuf, not gzip, but SOZip. #81
PostholerCom
started this conversation in
Ideas
Replies: 1 comment
-
@PostholerCom a pull request adding information about SOZip to the FGB page would be welcome. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In the guide it states:
_FlatGeobuf does not support compression while maintaining the ability to seek within the file. In particular, FlatGeobuf’s spatial index describes the byte ranges in the uncompressed file. Those byte ranges will be incorrect if the file is compressed.
A compression like gzip can be applied to the FlatGeobuf file in full, but keep in mind that storing the compressed file will eliminate random access support._
It refers to using gzip as an external alternative. You wouldn't want to use gzip on a .fgb file, you would want to use SOZip, search optimized zip This is a search optimized compression utility, making streaming and seek possible.
Additionally, if SourceCoop was to compress their .fgb files with SOZip those files would occupy roughly the disk space as their parquet equivalents.
Also of note, it is fully supported by GDAL, which is found under the hood of almost every open source project. SOZip works on .shp, .gpkg and .fgb, which have no internal compression.
Beta Was this translation helpful? Give feedback.
All reactions