Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Completely overhaul grid utils + min_Δx/y/z -> minimum_spacing + move x/y/zspacing to Grids #2991

Merged
merged 39 commits into from
Mar 24, 2023

Conversation

navidcy
Copy link
Collaborator

@navidcy navidcy commented Mar 21, 2023

This PR overhauls grid utils + adds minimum_x/y/zspacing methods that work on any grid.

(with @glwagner)

Closes #1119

@navidcy navidcy requested a review from glwagner March 21, 2023 02:27
@navidcy navidcy changed the title min_Δx/y/z -> minimum_spacing + move x/y/zspacing to Grids Completely overhaul grid utils + min_Δx/y/z -> minimum_spacing + move x/y/zspacing to Grids Mar 21, 2023
@navidcy
Copy link
Collaborator Author

navidcy commented Mar 21, 2023

@glwagner, I brought this commit of yours from #2867.
This was my first cherry-pick commit; thanks @angus-g!

@navidcy
Copy link
Collaborator Author

navidcy commented Mar 22, 2023

@glwagner I think there is still an issue in regrid_xy, only on GPU now.

@navidcy
Copy link
Collaborator Author

navidcy commented Mar 22, 2023

I think we can't call Nx_source_faces = size(source_grid, (Face, Center, Center), 1) inside a kernel...

@navidcy
Copy link
Collaborator Author

navidcy commented Mar 22, 2023

omg, perhaps I fixed it... see 5272106?

@navidcy
Copy link
Collaborator Author

navidcy commented Mar 22, 2023

now it's getting stuck on equality of two cuda subarrays... could you have a look at the GPU unit tests @glwagner?

@@ -41,7 +41,7 @@ function Base.:(==)(grid1::AbstractGrid, grid2::AbstractGrid)
x1, y1, z1 = nodes(grid1, (Face(), Face(), Face()))
x2, y2, z2 = nodes(grid2, (Face(), Face(), Face()))

return x1 == x2 && y1 == y2 && z1 == z2
CUDA.@allowscalar return x1 == x2 && y1 == y2 && z1 == z2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a nicer way to fix this --- this is sort of a hack that will be quite slow for large grids.

The nodes are using a view of an OffsetArray of a CuArray. From the stacktrace:

==(A::SubArray{Float64, 1, OffsetVector{Float64, CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}}, Tuple{UnitRange{Int64}}, true}, B::SubArray{Float64, 1, OffsetVector{Float64, CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}}, Tuple{UnitRange{Int64}}, true})

There's no GPU-friendly == operator for that deeply nested object. However, there are GPU-friendly operators for subarray.

I think when we return xnodes, etc it's probably nicer to use a plain view around the underlying array.

We can leave this for this PR and I can open a new PR to fix it later.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I was wondering yesterday about this.

@navidcy
Copy link
Collaborator Author

navidcy commented Mar 22, 2023

Seems likes docs are about to fail probably due to interior(ω_timeseries, :, 1, :, 3). So perhaps we fix the grid equality on this PR?

@navidcy
Copy link
Collaborator Author

navidcy commented Mar 22, 2023

Seems likes docs are about to fail probably due to interior(ω_timeseries, :, 1, :, 3). So perhaps we fix the grid equality on this PR?

I fixed everything so we are back on the 6hr30min train of building the docs....

if you wanna fix the grid equality sometime soon then go for it, otherwise let's leave it for a future PR...

@glwagner
Copy link
Member

Seems likes docs are about to fail probably due to interior(ω_timeseries, :, 1, :, 3). So perhaps we fix the grid equality on this PR?

ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mixing data types and instantiated types in the user interface
2 participants