question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`.write` does not save `None` values

See original GitHub issue

Description

When saving an AnnData object to disk, keys of a dictionary whose value is None seem not to be saved.

import numpy as np
import scanpy as sc
from anndata import AnnData

adata = AnnData(X=np.eye(3), uns={'key_1': 0, 'key_2': None})
adata.write('adata.h5ad')

_adata = sc.read('adata.h5ad')

gives

>>> _adata
AnnData object with n_obs × n_vars = 3 × 3
    uns: 'key_1'

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
LustigePersoncommented, Mar 28, 2022

Just for the sake of documenting this somewhere: I ran into this issue when I used the log1p function, which as a default writes {"base": None} to uns. However after saving and reloading the object, an error was thrown with rank_genes_groups (code), because it is looking for the base key which is not present anymore.

1reaction
WeilerPcommented, Jan 11, 2022

BTW, this is also an issue if you have None in one of your columns:

import numpy as np
import scanpy as sc
from anndata import AnnData

adata = AnnData(X=np.eye(3), uns={'key_1': 0, 'key_2': None, 'key_3': pd.DataFrame({'col_0': ['string', None]})})
# Alternative failure
# adata = AnnData(X=np.eye(3), uns={'key_1': 0, 'key_2': None}, obs={'col_0': [None]})
adata.write('adata.h5ad')
Traceback
Traceback (most recent call last):
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/utils.py", line 209, in func_wrapper
    return func(elem, key, val, *args, **kwargs)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/h5ad.py", line 270, in write_series
    group.create_dataset(
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/h5py/_hl/group.py", line 148, in create_dataset
    dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/h5py/_hl/dataset.py", line 140, in make_new_dset
    dset_id.write(h5s.ALL, h5s.ALL, data)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5d.pyx", line 232, in h5py.h5d.DatasetID.write
  File "h5py/_proxy.pyx", line 145, in h5py._proxy.dset_rw
  File "h5py/_conv.pyx", line 444, in h5py._conv.str2vlen
  File "h5py/_conv.pyx", line 95, in h5py._conv.generic_converter
  File "h5py/_conv.pyx", line 249, in h5py._conv.conv_str2vlen
TypeError: Can't implicitly convert non-string objects to strings

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/utils.py", line 209, in func_wrapper
    return func(elem, key, val, *args, **kwargs)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/h5ad.py", line 263, in write_dataframe
    write_series(group, col_name, series, dataset_kwargs=dataset_kwargs)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/utils.py", line 212, in func_wrapper
    raise type(e)(
TypeError: Can't implicitly convert non-string objects to strings

Above error raised while writing key 'col_0' of <class 'h5py._hl.group.Group'> from /.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_core/anndata.py", line 1912, in write_h5ad
    _write_h5ad(
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/h5ad.py", line 118, in write_h5ad
    write_attribute(f, "uns", adata.uns, dataset_kwargs=dataset_kwargs)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/functools.py", line 875, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/h5ad.py", line 130, in write_attribute_h5ad
    _write_method(type(value))(f, key, value, *args, **kwargs)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/h5ad.py", line 294, in write_mapping
    write_attribute(f, f"{key}/{sub_key}", sub_value, dataset_kwargs=dataset_kwargs)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/functools.py", line 875, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/h5ad.py", line 130, in write_attribute_h5ad
    _write_method(type(value))(f, key, value, *args, **kwargs)
  File "/miniconda3/envs/anndata_bug/lib/python3.8/site-packages/anndata/_io/utils.py", line 212, in func_wrapper
    raise type(e)(
TypeError: Can't implicitly convert non-string objects to strings

Above error raised while writing key 'col_0' of <class 'h5py._hl.group.Group'> from /.

Above error raised while writing key 'uns/key_3' of <class 'h5py._hl.files.File'> from /.

Though it does work for

adata = AnnData(X=np.eye(3), uns={'key_1': 0, 'key_2': None}, obs={'col_0': ['string', 'string', None]})
Read more comments on GitHub >

github_iconTop Results From Across the Web

`.write` does not save `None` values · Issue #673 - GitHub
Description When saving an AnnData object to disk, keys of a dictionary whose value is None seem not to be saved. import numpy...
Read more >
Writing "None" to a file in python - Stack Overflow
I'm parsing this file in Python, and I'm writing a file that every row in it will be a record (Comma separated) I've...
Read more >
Dealing with Missing Values NaN and None in Python - Medium
First of all, let's look at the data type of Python NaN and None . numpy.nan is IEEE 754 floating point representation of...
Read more >
Python | Remove None values from list - GeeksforGeeks
There can be multiple methods to remove None values from a Python list. ... We just check for not None values and construct...
Read more >
Write a data frame to a delimited file — write_delim • readr
needed - Values are only quoted if needed: if they contain a delimiter, quote, or newline. all - Quote all fields. none -...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found