[JOSS] Comments

Hi all!

I’m currently reviewing anndata’s submission to JOSS (openjournals/joss-reviews#4371). I think the paper is very well written and is worthy of publication at JOSS. However, in going over the reviewer checklist provided by JOSS, there are some minor outstanding issue which I cannot check off at the moment.

Below is an overview of all my relatively minor comments. I’m happy to discuss them in this thread or in a separate issue, if need be.

Robrecht

General issues

Repository It seems the repository has been renamed from theislab/anndata to scverse/anndata. Please fix this in JOSS.

Functionality

Performance “If there are any performance claims of the software, have they been confirmed?” There are various claims throughout the paper that anndata takes into account scRNAseq’s dataset sizes in terms of cpu and memory usage. For example: “Due to the increasing scale of data, we emphasized efficient operations with low memory and runtime overhead.” “In particular, AnnData takes great pains to support efficient operations with sparse data.” I totally agree with the authors on this matter. However, I don’t see any benchmarks in the paper or the documentation where anndata is compared to SCE, Seurat and/or loom in terms of memory usage, disk usage and/or execution time.

Documentation

Installation instructions I don’t see a “clearly-stated list of dependencies” in the documentation, as required by JOSS. I think the installation instructions could be expanded upon (e.g. https://scanpy.readthedocs.io/en/stable/installation.html).
Example usage “Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).” There are examples on concatenation and the on-disk format, but I don’t see any usage examples on related to the core API of anndata.
Community guidelines The contributing guidelines on readthedocs says “To be expanded.”

Software paper

State of the field The paper mentions “… where advances in single-cell RNA sequencing (scRNA-seq) gave rise to new classes of analysis problems with a stronger adoption of Python over the traditional R ecosystem.” Please provide a citation for this claim. https://www.scrna-tools.org/analysis (Zappia et al. 2018) seems to show the opposite.
Quality of writing The text in Figure 1 is too small to read when printed out on paper.
Quality of writing anndata is sometimes written as AnnData. I assume a distinction is made between anndata the package and AnnData the object, but in the text AnnData sometimes seems to refer to the object as well. Please clarify.
Quality of writing In the sentence “In the context of how (Wickham, 2014) recommends …”, I feel you should use an inline citation - that is @Wickham2014 instead of [@Wickham2014].
State of the field “For analysis of scRNA-seq data in R this has been further simplified by anndata2ri, which allows conversion to SingleCellExperiment and Seurat’s data format.” I think it’s also worth mentioning the ‘anndata for R’ on CRAN, which provides a reticulate wrapper to the anndata Python package. I believe this package has a more intuitive interface modifying anndata files when only small modifications need to be made or for people with anndata but not with SCE or Seurat.
State of the field If you want additional examples of anndata’s application domain, or, if you think mention of the following is an improvement to the manuscript: anndata is heavily used in OpenProblems, a community driven platform for benchmarking tasks in single-cell analysis (openproblems.bio). For example, in a NeurIPS 2021 competition (Luecken et al 2021, https://openreview.net/forum?id=gN35BGa1Rt) use it as the common data format throughout the whole benchmarking pipeline.