Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introductory tutorial similar to Rasterio's Python Quickstart #130

Merged
merged 7 commits into from
Sep 15, 2020

Conversation

pritamd47
Copy link
Contributor

Addressing the discussion at #40, this PR is an attempt to add a tutorial in the documentation, similar to Rasterio's Python Quickstart.

@coveralls
Copy link

coveralls commented Jul 10, 2020

Coverage Status

Coverage decreased (-0.3%) to 92.389% when pulling 94fd2ab on pritamd47:tutorial-docs into 47cc556 on yeesian:master.

@pritamd47 pritamd47 changed the title Introductory tutorial similar to Rasterio's Python Quickstart [WIP] Introductory tutorial similar to Rasterio's Python Quickstart Jul 10, 2020
@visr
Copy link
Collaborator

visr commented Jul 10, 2020

Awesome :)

Don't forget to add the new page here, it should probably go right below the index:

ArchGDAL.jl/docs/make.jl

Lines 11 to 12 in 47cc556

"index.md",
"GDAL Datasets" => "datasets.md",

Minor nit: could you use uppercase AG instead of ag as the module alias? It's consistent with the rest of the docs and Julia conventions where modules start with upper case.

For the picture, if possible, I'd rather not have it stored in the repository, to keep it as small as possible. Could you rebase this commit to remove it? Let me know if I can help. A solution could be to just post them in the thread here, which will give a GitHub link that we can paste in the markdown:

example

![example](https://user-images.githubusercontent.com/4471859/87169013-a32ee600-c2cf-11ea-9d09-e82446812282.png)

I believe another option would be to have Documenter generate the pictures by running the code, but I haven't looked into this option before.

@yeesian yeesian requested a review from visr July 12, 2020 21:58
@pritamd47
Copy link
Contributor Author

For the picture, if possible, I'd rather not have it stored in the repository, to keep it as small as possible. Could you rebase this commit to remove it?

This would be great! However I must admit, I do not understand the rebase thing you talked about. Will deleting the resources directory not suffice?

@visr
Copy link
Collaborator

visr commented Jul 15, 2020

If you delete the resources directory with git rm docs/src/resources/example.png, and add that as a new commit, the file will still be a part of the git history, making it bigger.

So to remove it from history, what you could do is git rm docs/src/resources/example.png, followed by git commit --amend. This modifies the commit you already made, instead of adding a new one. To push that here you need to do a force push, git push -f.

@pritamd47
Copy link
Contributor Author

Adding plot for documentation
Creating-Data

@pritamd47
Copy link
Contributor Author

Hey, I am writing the final portion of the tutorial, involving writing a dummy 2-D array to a raster dataset, but I am not sure if what I am doing is the best way to do so (Probably not).

So currently I am making the array (Z) following the exact operations here. Now, I am writing the array using these commands:

AG.create(
    AG.getdriver("GTiff"),
    width=240,
    height=180,
    nbands=1,
    dtype=Float32,
    filename="./temporary.tif") do dataset
        AG.write!(dataset, Z, 1)
end

This saves the array, but, is flipped around its Y axis. So to solve this, I flipped the array around its Y axis while writing, using AG.write!(dataset, reverse(Z, dims=1), 1); BUT, now the resultant raster file is flipped around the X axis 😞. Then, just out of curiosity, I used AG.write!(dataset, reverse(Z, dims=2), 1), and the resultant raster seems to be correct. I am confused about this behavior, and wanted to make sure that I personally understand it before writing any further.

Apart from this, I am also unsure about the best way to insert the transformation and projection information into the raster.

It would be great if you can provide an explanation of this. Thanks :)

@visr
Copy link
Collaborator

visr commented Aug 3, 2020

Sorry I forgot about this (feel free to ping again).

GDAL is row-major, whereas Julia is column-major. The raster data that you get from ArchGDAL, and are expected to pass to ArchGDAL, is permuted compared to what you may expect, so rows become columns and columns become rows. This is done to avoid creating copies on every read an write, which may hurt performance. See also the notes in http://yeesian.com/ArchGDAL.jl/latest/rasters/#Reading-Raster-Values-1 and discussion in #37 (I had exactly the same initial reaction as you). So perhaps you can first permutedims your array before passing it to ArchGDAL?

Apart from this, I am also unsure about the best way to insert the transformation and projection information into the raster.

I believe this should work with the setproj! and setgeotransform! functions, could you try out those? You can call getproj and getgeotransform to see what kind of data is expected.

@pritamd47 pritamd47 changed the title [WIP] Introductory tutorial similar to Rasterio's Python Quickstart Introductory tutorial similar to Rasterio's Python Quickstart Aug 20, 2020
@pritamd47
Copy link
Contributor Author

I think the PR is ready to be merged. Feel free to suggest any changes that you seem fit. @visr

@visr
Copy link
Collaborator

visr commented Aug 20, 2020

Cool! Could you perhaps try using this syntax: https://juliadocs.github.io/Documenter.jl/stable/man/syntax/#@repl-block? This is also used here for instance: https://raw.githubusercontent.com/yeesian/ArchGDAL.jl/master/docs/src/datasets.md

It means that you can remove the output from the document, as Documenter will run the examples and paste in the output as the documentation builds. It's good to build the docs locally to confirm it is working as intended.

@pritamd47
Copy link
Contributor Author

From what I am understanding, @repl or @example will evaluate the script. However, for the first half of the documentation, I read in a LandSat image (quite huge, ~66MB). If there will be no such file (since such a large file cannot be included in the package) in the working directory, the evaluation should result in an error. Is there a way to download the file on-the-fly, so that the code blocks can be evaluated?

@visr
Copy link
Collaborator

visr commented Aug 20, 2020

Yep, there is, and after #138 was fixed this works directly now:

ds = ArchGDAL.read("/vsicurl/http://landsat-pds.s3.amazonaws.com/c1/L8/037/034/LC08_L1TP_037034_20160712_20170221_01_T1/LC08_L1TP_037034_20160712_20170221_01_T1_B4.TIF")

For reading rasters, we could also use the new readraster, see https://yeesian.com/ArchGDAL.jl/dev/rasters/#The-RasterDataset-type. This could make the example a bit easier. Or we can at least make a mention of it.

Side note, when typing macro names in GitHub it is best to quote them with backticks like @example, to avoid pinging unrelated people.

@pritamd47
Copy link
Contributor Author

@visr please review the changes when you get the time. Thanks :)

@yeesian yeesian merged commit 0c57b98 into yeesian:master Sep 15, 2020
@yeesian
Copy link
Owner

yeesian commented Sep 15, 2020

This is awesome, thank you so much! I am really sorry it took me so long to get around to review this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants