combat processing errors

…when I did combat batch correction, it outputs errors

# ComBat batch correction
sc.pp.combat(adata, key='sample')

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-350690ae55dc> in <module>
      1 # ComBat batch correction
----> 2 sc.pp.combat(adata, key='sample')

~/anaconda2/envs/scanpy/lib/python3.6/site-packages/scanpy/preprocessing/_combat.py in combat(adata, key, covariates, inplace)
    266         denom =  np.dot(dsq, np.ones((1, n_batches[j])))
    267         numer = np.array(bayesdata[batch_idxs] - np.dot(batch_design.loc[batch_idxs], gamma_star).T)
--> 268         bayesdata[batch_idxs] = numer / denom
    269 
    270     vpsq = np.sqrt(var_pooled).reshape((len(var_pooled), 1))

ValueError: operands could not be broadcast together with shapes (23259,18243) (23259,15479) 

Versions:

scanpy==1.4.6 anndata==0.7.1 umap==0.3.10 numpy==1.17.3 scipy==1.3.1 pandas==0.25.2 scikit-learn==0.21.3 statsmodels==0.11.1 python-igraph==0.8.0

[ivirshup]

Thanks @rika-N! That’s very helpful in figuring this out. I can now reproduce this with:

import scanpy as sc
import numpy as np
import pandas as pd

a = sc.AnnData(
    np.random.random((5, 5)),
    obs=pd.DataFrame({"batch": list("aaabb")}, index=list("11234"))
)
sc.pp.combat(a, "batch")

Traceback

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-16-e35e04bc067c> in <module>
      7     obs=pd.DataFrame({"batch": list("aaabb")}, index=list("11234"))
      8 )
----> 9 sc.pp.combat(a, "batch")

~/github/scanpy/scanpy/preprocessing/_combat.py in combat(adata, key, covariates, inplace)
    266         denom =  np.dot(dsq, np.ones((1, n_batches[j])))
    267         numer = np.array(bayesdata[batch_idxs] - np.dot(batch_design.loc[batch_idxs], gamma_star).T)
--> 268         bayesdata[batch_idxs] = numer / denom
    269 
    270     vpsq = np.sqrt(var_pooled).reshape((len(var_pooled), 1))

ValueError: operands could not be broadcast together with shapes (5,5) (5,3) 

The issue is that the combat function currently does not work when the observation names are not unique.

[rika-N]

@Zifeng1995, @ivirshup

I received the same error caused by wrong shape of “numer” in line 268 in _combat.py. In my case, I could resolved this problem by generating unique cell names ( i.e. adata.obs_names) as following.

Workaround Before combat execution, in concatenate process of batch data, I specified index_unique=‘-’ . e.g.) adata1.concatenate(adata2, adata3, …, index_unique=‘-’)

When index_unique=None, the error was occurred. However index_unique=‘-’ was fine in my case. I’m afraid that couldn’t trace root cause of this problem due to busy day, but I hope this helps you.

Sincerely.