indexing of AnnData

The task is to write the following preprocessing sequence using an AnnData instance adata.

meanFilter = 0.01
cvFilter = 2
nr_pcs = 50

ddata = adata.to_dict()
X = ddata['X']
# row normalize                                                                                                                                                                  
X = row_norm(X, max_fraction=0.05, mult_with_mean=True)
# filter out genes with mean expression < 0.1 and coefficient of variance <                                                                                                      
# cvFilter                                                                                                                                                                       
X, gene_filter = filter_genes_cv(X, meanFilter, cvFilter)
# compute zscore of filtered matrix                                                                                                                                              
Xz = zscore(X)
# PCA                                                                                                                                                                            
Xpca = pca(Xz, nr_comps=nr_pcs)
# update dictionary                                                                                                                                                              
ddata['X'] = X
ddata['Xpca'] = Xpca
ddata['var_names'] = ddata['var_names'][gene_filter]
sett.m(0, 'Xpca has shape',
    ddata['Xpca'].shape[0], 'x', ddata['Xpca'].shape[1])
from ..ann_data import AnnData
adata = AnnData(ddata)
print(adata.X)

While the previous snippet works just as expected, when I want to do the same without a ddata object, some uncontrolled behavior comes up. Indexing doesn’t work as expected anymore. @flying-sheep: could you have a look at why adata['Xpca'] = Xpca in the following throws an

>>> adata['Xpca'] = Xpca
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

in the following snippet

X = adata.X
# row normalize                                                                                                                                                                  
X = row_norm(X, max_fraction=0.05, mult_with_mean=True)
# filter out genes with mean expression < 0.1 and coefficient of variance <                                                                                                      
# cvFilter                                                                                                                                                                       
X, gene_filter = filter_genes_cv(X, meanFilter, cvFilter)
# compute zscore of filtered matrix                                                                                                                                              
Xz = zscore(X)
# PCA                                                                                                                                                                            
Xpca = pca(Xz, nr_comps=nr_pcs)
# update adata                                                                                                                                                                   
adata.X = X
adata = adata.var_names[gene_filter] # filter genes                                                                                                                              
adata['Xpca'] = Xpca
sett.m(0, 'Xpca has shape',
    adata['Xpca'].shape[0], 'x', adata['Xpca'].shape[1])
print(adata.X)

I played around quite some bit, but the only solution that I got running then had the numerically incorrect result. It’s quite to hard to keep this sequence of steps nicely organized.

PS: the snippet appears in scanpy/preprocess/advanced.py and an example would be ./scanpy.py nestorowa16 diffmap -r pp.

[falexwolf]

ok, of course this right, let’s discuss in person.

[flying-sheep]

something like adata.var = adata.var[gene_filter] should work

i disagree. adata = adata[:, gene_filter] should work. the object shouldn’t be able to be in an invalid state.