ENH: Add lock kwarg to to_raster and add examples of using dask for file io

[snowman2]

• Closes BUG: Image missing tiles with dask write #220
• Tests added
• Fully documented, including docs/history.rst for all changes and docs/rioxarray.rst for new API

cc: @djhoese

[codecov]

Codecov Report

Merging #223 (34764e3) into master (7e4a87f) will decrease coverage by 1.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #223      +/-   ##
==========================================
- Coverage   93.17%   92.13%   -1.04%     
==========================================
  Files          11       11              
  Lines        1260     1259       -1     
==========================================
- Hits         1174     1160      -14     
- Misses         86       99      +13     

Impacted Files	Coverage Δ
rioxarray/raster_array.py	98.12% <ø> (ø)
rioxarray/raster_dataset.py	100.00% <ø> (ø)
rioxarray/raster_writer.py	77.50% <100.00%> (-16.33%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7e4a87f...34764e3. Read the comment docs.

[djhoese]

I just found out that this exists and could maybe simplify this:

https://github.com/dask/dask/blob/f41325512a4251dbaa7f7c781414d913066a6c34/dask/utils.py#L1017-L1028

I found that some of the dask functions actually use this and default to it.

[snowman2]

@djhoese that looks pretty neat. However, that won't work for to_raster. The reason being that we need a global lock as only one process can write to the file at a time ref.

[djhoese]

Isn't that how this to_hdf function is using it: https://docs.dask.org/en/latest/dataframe-api.html#dask.dataframe.DataFrame.to_hdf

[snowman2]

Isn't that how this to_hdf function is using it

For GeoTiff you need the global lock or you get the problems you found in #220

[djhoese]

So "global" as in the global scope of the calling module, right? Couldn't users use this utility function so they don't have to decide what lock to choose?

[snowman2]

So "global" as in the global scope of the calling module, right?

It means that the lock needs to be shared by all processes and threads.

Couldn't users use this utility function so they don't have to decide what lock to choose?

I don't think so. Based on the dask issues I have seen, the SerializableLock only locks the threads within the process that contains. And the multiprocessing lock doesn't work for a cluster of machines. So, the dask Lock is the only option. But, I recommend trying it out and seeing what happens. You should be able to try it out now and see if it works or not.