Reuse orbital products in density calculation for multiple DMs

If one wants to calculate the LDOS(E) from a set of density matrices for different energies, one needs to call dm.density() for each energy, which results in the computation of the orbital products for each energy. This makes it very difficult to compute the energy resolved LDOS in practice.

It would be nice if the orbital products could be calculated once and then used for all density matrices. The two naive approaches would be:

1. Save orbital products and pass them to density calls. Seems unfeasible because of the memory required, since the shape of the products would be (grid_x, grid_y, grid_z, orb_pair), being orb_pair n_orbitals ** 2 / 2 (I think). Although perhaps they could be stored in a sparse manner and then it would be doable (?).
2. Have a density function that accepts multiple DMs and computes all the densities at the same time. If you need to calculate the density for the whole grid, this would also require a lot of memory: (grid_x, grid_y, grid_z, nE).

In my particular case, I want to average all grids on X and Y, ending with a final array of (grid_z, nE), so perhaps there is some way of avoiding the massive memory requirements?

As a note, calculating STS spectra is nice to compare calculations with experiments. These spectra are basically LDOS(x, E), being “x” some position in a path of interest. Currently, there is no way to project real space quantities into a path or even a generic collection of points in sisl. One has to go through a grid, which makes it very computation and memory demanding. Wouldn’t it be nice to have something like this? If so I can open a separate issue. I just brought it up here because if you only need to compute the LDOS along a path then both options would be reasonably cheap.

[pfebrer]

I have already a first working implementation of precalculating orbital values in python + computing the product with cython. In terms of computation time, the implementation seems to be between 1 and 2 orders of magnitude faster than the current one. However not all the gain comes from the different approach and cython, I have found some parts of the current code that are slowing the current implementation unnecessarily (I think). I will submit the PR and discuss what I have found later today.

[zerothi]

1. No they don’t.
2. I tested quite a bit of how to do this, and this seemed like the fasted way

I have tried many things in this routine to see how Python could be as fast as possible, it wasn’t easy. For 1, I can’t recall why, probably there was something I thought about, but I can’t remember. As for 2, the problem is that you want to remove the points, and doing:

idx = idx[idx[:, 0] >= 0, 0]
...

proved much slower (depending on the grid-sampling). I can’t recall if I tried:

filter = np.logical_and(0 <= idx[:, 0], idx[:, 0] < shape[0])
filter = np.logical_and(filter, 0 <= idx[:, 1])
...

but you are welcome to try out and see if it is faster, proper tests are really required since the complexity makes it hard to know which calculation is the most optimal.

As for having a failed point. It should only amount to a single point that if is in, should be so far off that the values are 0 anyways. But, I get your drift, it really shouldn’t be there. It was a performance wise work-around.