WIP branch for implementing ABCLattice change.

[tuopuu]

Lattices with primitive vector representation

The main reason to moving from base vector representation to primitive vectors is that it allows defining the type of the lattice more conclusively. In a typical use-case, the user would call new ABCLattice methods that replace old factory functions to create a specific lattice.
There are some points that I would like to make, that give some reasoning to the implementation proposed here:

Firstly, calculation of primitive vectors based on lattice constants a,b,c and the angles between vectors alpha, beta, gamma would have to be done separately for all supported lattice types in their respective factory functions. However, there is a formula (the triclinic one) that can be used for generating the primitive vectors for all lattice types, with a catch, that it will be hard to make (float) comparisons of the primitive vectors of different lattices. Typically, in LB-codes lattice nodes are compared with respect to their indices, so that inaccuracy e.g. get_coordinates is not very relevant. I think that using the same formula for every lattice type is a good compromise because it significantly reduces the amount of code. Calculation of primitive vectors can be added inside ABCLattice as a concrete method, so that it is accessible from all child classes later.

Secondly, constructors of Mesh and Particles classes currently generate empty container, but Lattice ctor generates a full lattice, based on the given size,origin,etc parameters. To that end, Lattice (and H5Lattice with its create_new classmethod) needs a big list of parameters, contrary to Mesh and Particles ctors that only need one, the name. I suggest the following: XLattice ctor only takes as parameters the ones that are specific to the implementation. For examples, Lattice always needs a name, but H5Lattice only needs a group, and not a name. Separating specific functionality to XLattice ctor allows implementing many shared functions of the lattices in ABCLattice as concrete methods and XLattice would inherit all that from there.
I think that technically they would have to be regular def create_xxx_3d(self, ...) style methods so that they would work correctly when called from within a child class, such as H5Lattice. Including create_xxx_3d methods with ABCLattice has a benefit that it would provide factory method functionality to the other lattice classes as well, instead of only the Lattice.
I propose that lattice classes would work the following way:

A = Lattice('somename') 

only sets the name of the lattice

B = Lattice('somename').create_cubic_3d( (10,20,30), (0,0,0), a)

first creates the lattice base class with name 'somename'. ABCLattice.create_cubic_3d calls ABCLattice constructor, which sets the parameters a,b,c,etc. accordingly, calls ABCLattice._set_primitive_vectors, and finally calls Lattice._create_new which finally generates the lattice table.

C = H5Lattice(group)

Check if group already has a lattice stored in it. If yes, then generate H5Lattice from the stored information. If no, then just store group to self._group.

C = H5Lattice(group).create_cubic_3d( (10,20,30), (0,0,0), a)

ABCLattice.create_cubic_3d calls ABCLattice ctor with correct parameters, sets primitive vectors, and finally calls H5Lattice._create_new which writes a,b,c,etc attributes to the file/group pointed by group, and also generates Pytables arrays for storing data.

[tuopuu]

@kemattil, @itziakos, @roigcarlo, @nathanfranklin This proposal is now open for discussion. Please, take a look when you got time.

[kemattil]

@tuopuu, I defnitely would like to review and comment on this. I'm currently (trying) to have my vacation. I hope this can be kept open until I return to office (around mid-August) ...

[itziakos]

alternative constructors are commonly defined as classmethods

@classmethod
def create_cubic_3d(cls, size, origin, a):
        """
        Creats a cubic lattice
        """
        primitive_vectors = calculate_primitive_vectors(a, a, a, 90.0, 90.0, 90.0)
        return cls(size, origin, primitive_vectors, kind=LATTICEKIND.CUBIC_3d)

And there is no need for create_new

[tuopuu]

I couldn't make using classmethod to work with all the differences between H5Lattice, Lattice (and proxylattice). H5Lattice has the group parameter which Lattice doesn't have, and also the way of generating the table for data is different in these both. Proxylattice has it's own specific features with the reference pointer to external data.

Therefore, distinct functionality has to be stored in methods inside a child class of Abstractlattice, and these methods have to be somehow called from Abtractlattice "alternative" constructors such as create_cubic_3d. I don't see a way to do it if classmethod is used.

[itziakos]

H5Lattice is to be initialized only inside the H5CUDS file please see comment (#191 (comment))

It does not make sense for the user to be allowed to create HDF5 lattice backed files on an arbitrary group. Do you have a usecase where this is necessary?.

[tuopuu]

H5Lattice is to be initialized only inside the H5CUDS file please see comment

You're right. We don't have a usecase to use other groups. This is a good point that can potentially simplify things a lot.

[tuopuu]

However, for lattice-boltzmann HPC applications, we should be able to construct hdf5-backed lattice without generating the whole lattice in memory first, because often the lattice is much bigger than can be fit into a single PC memory. This doesn't require the use of 'group' parameter, but we should still allow generation of hdf5 backed lattice somehow, because currently it's not supported inside H5CUDS (that only has get.../add... methods for CUDS objects).

[itziakos]

Yes, that is a valid usecase, which we should address, (by either by providing a create_dataset or my preferred option using preferably the 'sparse' dict based allocation), but not by mangling the initialization of python objects :P

[itziakos]

B = Lattice('somename').create_cubic_3d( (10,20,30), (0,0,0), a)

why not Lattice.cubic_3d(name, parameters)

[itziakos]

Secondly, constructors of Mesh and Particles classes currently generate empty container, but Lattice ctor generates a full lattice

This is not necessary to happen like this, it is the current native Lattice implementation that requires a full Lattice not the api. One can always:

• use sparse arrays to do the same job.
• use default dicts with key the index tuple instread of a numpy array
• Create the numpy array lazily when the first access (get_node, update_node) happens is also easily done in order to avoid allocating memory when all what people want is an empty Lattice.

C = H5Lattice(group)

Creating an H5Lattice on its own is not something that we should provide. It would be better if we provide a create_dataset method on the H5CUDS file and in the various engines. Or create an empty Lattice as proposed earlier. In sort the hdf5 cuds container objects are not to be used initialized outside the H5CUDS file object

[tuopuu]

Thank you, @itziakos, for the comments. I'm not sure if I understand correctly how your way of implementing primitive vectors would work for all lattices, but I'll give it a try.

We also had a long discussion with @kemattil about two weeks ago, and he also had good ideas about how to change this.

[kemattil]

I think there are now so many aspects/issues interlinked that we need to simplify things a little bit.

Therefore I made a new proposal for implementing all 3D Bravais lattices (see the branch feature-lattice-prim-cell 4d40c31). This proposal will result in small/moderate changes to existing code and to a simple class design/hierarchy.

Main features (files primitive_cell.py, abc_lattice_new.py, and lattice_new.py):

• a data structure for representing primitive cells of a lattice (class PrimitiveCell)
• ABCLattice api changed, i.e. properties type and base_vect replaced with prim_cell
• standalone class Lattice takes a reference to PrimitiveCell at the initialization
• get_coordinates implemented in ABCLattice; uses the primitive vectors from PrimitiveCell (properties p1, p2, and p3)
• the factory functions make_XXX_lattice call the PrimitiveCell class methods create_cell_XXX_lattice
• the H5Lattice api remains almost intact, i.e. class method create_new should take prim_cell instead of type and base_vect, and the primitive cell should be read and stored in I/O

The issue of handling very large lattice can be discussed separately.

[tuopuu]

@kemattil, please, make a WIP pull request so we can start discussing your proposal.

[kemattil]

WIP pull request made, #205

[tuopuu]

Primitive vector representation will be implemented in PR #205. I'll close this one.