Multimesh gap and mrg tree

[mqiyang]

Hi, I have a question about the adjacency of the multimesh. For the mesh of domain0, the meshes start from 0 to M, if I make the mesh of domain1 start from M+1, then there will be a gap between the domains when visualized in VisIt. The way that I tried to shun this problem is making domain1 starts from M so that it overlaps with domain0. It works as expected but I suspect that it may cause some discontinuity in post-processing. My data is structured, node-centered quadmesh.

1. Is manually making adjacent blocks overlap a proper way to fix the gap problem?
2. If not, after reading through the manual books, I can think about two possible ways: ghost-zone and MRG tree of connectivity. It seems that the adding the ghost-zone only works for zone-centered mesh and cannot be applied to my case. Is it right?
3. I have thoroughly read the section about mrg tree, but I'm still very confusing. In my understanding, After DBMkMrgTree, I can use DBAddRegion to add the nodes. However, I am still struggle to understand how to create groupel maps and how the segments of the node different from those of a groupel map. I looked through the previous answers related to this topic but the links all expired. Is there any code examples that can help me solve the problem?

Thanks a lot.

[markcmiller86]

@mqiyang very sorry for delay in responding. Lemme see if I can help with some of your questions...

Hi, I have a question about the adjacency of the multimesh. For the mesh of domain0, the meshes start from 0 to M, if I make the mesh of domain1 start from M+1, then there will be a gap between the domains when visualized in VisIt. The way that I tried to shun this problem is making domain1 starts from M so that it overlaps with domain0. It works as expected but I suspect that it may cause some discontinuity in post-processing. My data is structured, node-centered quadmesh.

Hmm...are you using the DBPutMultimeshadj() call for this or Mesh Region Group (MRG) trees? Your headline suggests mrg trees but your use of the word "adjacency" here suggested maybe not.

1. Is manually making adjacent blocks overlap a proper way to fix the gap problem?

Well, short answer is yes. But, it also depends. For structured meshes, VisIt does have the ability to generate this information while it is reading data in. For UNstructured meshes, the database itself must resolve the overlap for VisIt. You mention you have a structured mesh. So, in theory, VisIt can compute this for you. In VisIt parlance, these extra zones to resolve overlap are called ghost zones. Another name for ghost is halo. The idea is these are kinda sorta extra zones/cells in the mesh. They are there for proposes of computing things (like normals for shading or derivatives or getting contouring correct) but are not intended to actually be displayed as part of the final result. They are there but you can't see 'em. They are ghosts. Some other refs...

• https://blogs.helsinki.fi/computational-field-theory/2018/01/11/visit-and-ghost-zones/
• https://www.kitware.com/ghost-and-blanking-visibility-changes/

Now, a common issue with structured grid in parallel is that two neighboring pieces include only the nodes up to their adjoining boundary. Niether piece has any nodes in common along that boundary. That will lead to gaps. So, both pieces that share a boundary also need to share the nodes on that boundary. And, those shared nodes are NOT GHOSTS. They are displayed in every view of the data on both pieces. This is true regardless of mesh type (structured or UNstructured) and VisIt cannot fill such a gap itself. Sometimes, both pieces need to share multiple layers of nodes (or zones). For example, to compute 2nd order derivatives at the nodes on a piece boundary, two-layers of ghost nodes are required, etc.

For strong scaling scenarios where you increase the number of pieces and each piece gets smaller and smaller, the relative cost of the ghost (or halo) data gets to be hugely prohibitive which is why not having the data producer write it and instead have the consumer produce it on the fly is sometimes very important.

2. If not, after reading through the manual books, I can think about two possible ways: ghost-zone and MRG tree of connectivity. It seems that the adding the ghost-zone only works for zone-centered mesh and cannot be applied to my case. Is it right?

No. VisIt handles both ghost nodes and ghost zones. In Silo's DBPutXxxmesh() calls, you will find DBoptlist options for things called hi and lo offsets. These options allow the writer to indicate what portions of a mesh piece are real and what portions are ghost. Indeed these are always associated with the mesh coordinate data (so node-ish, right) But, if you specify two layers of ghost nodes for example, that implies there are zones/cells consisting entirely of only ghost nodes and those are ghost zones and the associated zonelist (for UNstructured cases) or zone-centered variables will need to contain data for those entries as well.

3. I have thoroughly read the section about mrg tree, but I'm still very confusing. In my understanding, After DBMkMrgTree, I can use DBAddRegion to add the nodes. However, I am still struggle to understand how to create groupel maps and how the segments of the node different from those of a groupel map. I looked through the previous answers related to this topic but the links all expired. Is there any code examples that can help me solve the problem?

Ok, so indeed you are using MRG trees. But, I don't think you need to for the special case of ghosting. Try using the DBoptlist options for hi and lo offsets for this. I could make this clearer in Silo's documentation too.

[mqiyang]

Sorry for my belated reply and confusion. Let me rephrase my question.
I have a volumetric data set and I divided them in to domain along x-direction with grids range [1,n]. The data is structured, node-centered quadmesh.

1. if I set:
   for domain0: the grids range [1, m]
   for domain1: the grids range [m, n]
   then I got a decent plot. (the plot and below are pseudo-color sliced at z-direction)

2. if I set:
   for domain0: the grids range [1, m]
   for domain1: the grids range [m+1, n]
   then my plot has a gap between the two domains:
   

3. if I set ghost offset on (2) with only setting the hi_offset.
   for domain0: the grids range [1, m];
                         lo_offset(x, y, z) = 0;
                         hi_offset(x)=1 and hi_offset(y, z)=0.
   for domain1: the grids range [m+1, n];
                         lo_offset(x, y, z) = 0;
                         hi_offset(x, y, z) = 0
   the the gap in between the domains became larger.
   

4. then I tried set ghost offset on (1) with only setting the lo_offset:
   for domain0: the grids range [1, m];
                         lo_offset(x, y, z) = 0;
                         hi_offset(x, y, z)= 0;
   for domain1: the grids range [m, n];
                         lo_offset(x) = 1 and lo_offset(y, z)=0
                         hi_offset(x, y, z) = 0
   the plot I got is exactly the same as (2).
   

As I mentioned before, I think the approach (1) may lead problems when doing some post-processing. You mentioned above the structured quadmesh data should be well-handled by VisIt, but I still would like to find a safer and more versatile way.

[markcmiller86]

Hmmm...VisIt cannot create any mesh (or quad elements) for you. It can only know and record that some nodes and/or elements in one piece are copies (e.g. ghosts) of nodes and/or elements in other pieces. So, you can't have two pieces that don't have at least some copies of other piece's elements (in your case copies of nodes on the boundaries would suffice unless you plan on doing contour plots, then you want copies of whole quad elements 1 layer deep on each piece). Abutting pieces must at least share nodes otherwise, you will have gaps because VisIt cannot take nodes on the boundary of one piece and nodes on the boundary of some other piece and connect them to create quads that fill in the gap. First, how would VisIt know what pieces are supposed to connect to which other pieces and, in particular, which nodes on those pieces. The only solution is for all pieces to have copies of the the things they share.

Does that help?

[mqiyang]

Thank you for your clear explanation. It's really helpful for me to solve this problem. If I'm understanding correctly, it means I don't need to set any ghost offset and just implement approach (1), where domain0 and domain 1 share the boundary data at the grid m?

[cyrush]

@mqiyang Yes, I believe that is correct.

[mqiyang]

Thank you for your reply. My question is solved.