Issue generating Brueckner orbitals from CCSD

Hi

I am trying to build a program to generate Brueckner orbitals in PySCF via CCSD. However, my PySCF results do not align with ACES2 results starting at the second iteration. At this point, I start to see slight deviations in the energy that increase in magnitude as I continue to iterate, my T1 norms stop decreasing, and the second order amplitudes yield an MP2 energy that is appreciably different than ACES2. All that to say ACES2 converges, while my implementation does not.

After I rotate the occupied and virtual orbitals with T1, I orthogonalize the set of C, construct a DM, feed that into the SCF class without iteration to rebuild the Fock matrix, re-transform 2 e- integrals, and then finally re-run CCSD. I feel like there is something that I am not seeing in my PySCF implementation, or there is something in my code I am overlooking. I was hoping someone could lend me some help.

Disclaimer that there is a related request open at #1343, but I have not been able to track down any implementation to compare my code to. Perhaps you can direct me in this regard, @zhcui ? To conserve space, I have only posted the functions that drive the brueckner routine, which rotates orbitals and orthogonalize them., but will attach the entire python script for reference.

def orthogonalizeMO(mf,C):
    ovrlap=mf.get_ovlp()
    ovrlapMO=(C.T@ovrlap)@C
    roots,vecs=np.linalg.eigh(ovrlapMO)
[bruecknerCCSD.py.zip](https://github.com/pyscf/pyscf/files/9913594/bruecknerCCSD.py.zip)
[bruecknerCCSD.py.zip](https://github.com/pyscf/pyscf/files/9913596/bruecknerCCSD.py.zip)

    indx = roots.argsort()
    vecs = vecs[:, indx]
    roots=roots[indx]
    newroots=np.zeros((len(roots),len(roots)))
    for i in range(len(roots)):
        newroots[i][i]=1.0/np.sqrt(roots[i])

    X=(vecs@newroots)@vecs.T
    orthog_C=C@X
    print('Verify C^tSC==1',np.allclose((orthog_C.T@ovrlap)@orthog_C,np.eye(X.shape[0])))
    return orthog_C


# Performs Brueckner rotation on occ/virt orbitals
def T1rotate(coeff,t1):
    newcoeff=coeff
    nocc,nvirt=t1.shape

    C_occ=coeff[:,:nocc]
    C_virt=coeff[:,nocc:]
    C_virtold=C_virt

    C_virt=C_virt-C_occ@t1
    C_occ=C_occ+C_virtold@t1.T

    newcoeff=np.zeros(coeff.shape)
    newcoeff[:,:nocc]=C_occ
    newcoeff[:,nocc:]=C_virt

    return newcoeff


def run_brueckner(mf,t1,coeff,nocc,mo_occ,t1TOL=10E-4):
    t1norm=np.linalg.norm(t1)
    newC=coeff
    for i in range(4):
        print('Norm of t1:',t1norm)
        newC=T1rotate(newC,t1)
        newC=orthogonalizeMO(mf,newC)

        # Rebuild Fock matrix
        mf=scf.RHF(mol)
        dm1=mf.make_rdm1(newC,mo_occ)
        mf.init_guess=dm1
        mf.run(max_cycle=0)

        # Converge CCSD eqns & extract new T1
        mycc=cc.CCSD(mf)
        eris=mycc.ao2mo()
        mycc.kernel(eris=eris)
        t1=mycc.t1

        print('max of t1: ',np.max(np.abs(t1)))
        t1norm=np.linalg.norm(t1)

    return mf,dm1

[zww-4855]

No problems at all, @pwborthwick , I have learned something new.

@maxnus I missed this comment earlier: We have found that including nonlinear terms in the series do appreciably impact convergence rate. So in practice, we truncate the expansion at (1+T1).

I tested @zhcui implementation, where both ACES2 and pySCF converge toward the same CCSD energy. One thing I found interesting is that his implementation also rotates the initial T1 and T2 amplitudes at each Brueckner iteration. To my knowledge, ACES does no such thing. I have not been able to convincingly explain why my python implementation fails; I am assuming I have overlooked some formal technicality in python and/or PySCF. I will gradually look more into this.

In the end, this implementation converges faster than ACES for H2O. I expect this to generally be the case for arbitrary systems as well, since ACES does not employ DIIS for Brueckner.

I appreciate the commentary everyone. Take care.

Zack

[zhcui]

@zhcui , I’m curious why you have a custom logm routine rather than using scipy.linalg.logm? As far as I can see there is no difference to speed or numerical accuracy when using scipy. Peter

Hi Peter, scipy’s logm may return a complex matrix. Best, Zhihao

You could use np.real(scipy.linalg.logm) save you a subroutine. Best Peter

Try la.logm(np.eye(2) * -1)

Sorry not to get back sooner - it was night on this side of the Atlantic. Before I try np.eye(2)*-1 what I expect is since ln(A) is diagonal if and only if A is diagonal I expect scipy to produce a diagonal matrix with diagonal elements ln(-1) = ln( eiπ ) = iπ. The real part of which is a matrix entirely of zeros.,this is exactly what it produces (with some numerical noise). However, the original matrix has negative real eigenvalues so there doesn’t exist a principal logarithm and the logarithm is not unique. Your algorithm is better as it finds a non-zero matrix among the possible solutions (determinant still zero), which is essentially lemma 2 from Sherif and Morsy I take your point then that there are (pathalogical?) situations that can occur where scipy will fail and your (rather neat) algorithm will not. Thank you for pointing out the potential problems with the scipy logm - When I find a case in practice where scipy fails I’ll report back.

PS. The most ‘pathalogical’ case I’ve found so far is H2 in STO-3G where the initial rotation matrix is I2 and both scipy and your algorithm return a 2x2 matrix of zeros. The iterations converge after 1 iteration to the same value as with diis switched off.

Ok, sounds great.