Error with launching cellxgene after manually adding metadata to AnnData file (.h5ad)

Hi,

I have an AnnData file adata.h5ad file with certain columns in adata.obs. I have some more metadata that I would like to add to adata.obs. I follow some simple steps and recommendations provided online to add these columns to adata.obs

# df_annot - pandas dataframe with annotations/columns I would like to add
# df_annot has identical index as adata.obs
adata.obs = pd.concat([adata.obs,df_annot],axis=1)

Note: I am using cellxgene==0.14.1 The launching of original adata.h5ad using cellxgene works using the following command - cellxgene launch adata.obs --port 8000 --disable-diffexp

However, when I try to launch adata.h5ad after having added additional columns (metadata) to adata.obs using the same command, I get the following error -

Error: 'dict' object has no attribute 'dtype' - file not found or is inaccessible.  File must be an .h5ad object.  Please check your input and try again.

Initially, I thought there might be an issue with how I am adding metadata/columns to AnnData file - adata.obs. I explored further by re-reading the modified h5ad file using scanpy (sc.read_h5ad) and processing it further by performing steps like ranking gene groups (sc.tl.rank_genes_groups). These steps were successful, so it seems unlikely that this is an AnnData (.h5ad) file issue.

Would love to hear your thoughts on where the issue might be arising from and what would be the best way to debug and getting it to work.

cellxgene installed in a conda environment with the following dependency versions -

python==3.7.3
numpy==1.16.3
pandas==0.24.2
numba==0.43.1

cellxgene==0.14.1
anndata==0.6.22post1
scanpy==1.4.5.1

Update (Mar 10, 2020):

To clarify, I am using different environments for cellxgene, and processing the AnnData file. CELLXGENE CONDA ENVIRONMENT

python==3.7.3
numpy==1.16.3
pandas==0.24.2
numba==0.43.1
cellxgene==0.14.1
anndata==0.6.22post1

ANNDATA PROCESSING CONDA ENVIRONMENT

python==3.8.1
numpy==1.18.1
pandas==1.0.1
numba==0.48.0
anndata==0.7.1
scanpy==1.4.5.1

[bkmartinjr]

I believe I understand the issue for @GMaciag, and it may very well be the issue for @shahnirav1005

Background: scanpy and anndata provide backward compatibility, but not forward compatibility. H5AD created by old anndata versions can be read by newer versions, but not vice versa.

In other words:

• H5AD created with anndata version “N” are readable by anndata version “N+1” (and later)
• H5AD created by anndata version “N+1” are not normally readable by anndata version “N”

In @GMaciag case, you are using a more recent anndata to create the data, and reading it with an old one.

cellxgene 0.14.1 mandates a slightly older version because the latest anndata was incompatible with cellxgene. So we pinned the version dependency.

You can fix this in two ways:

1. Create the H5AD using the 0.6.22post1 version of anndata (and correspondingly older scanpy)
2. Wait a few days, and we will release a new version of cellxgene that fixes this (works with the latest anndata/scanpy).

Apologies for the inconvenience!

[bkmartinjr]

We will announce the release on the CZI Science slack site (#cellxgene-users channel)