Fails with "not enough empty droplets" error even though --low-count-threshold 2

CellBender fails on my data with “not enough empty droplets” error even though I set --low-count-threshold to 2. The error & log itself are a bit weird (look at the scale of nUMI filtered). I get this exact error (with different UMI numbers) on 2 very different datasets.

cellbender:remove-background: 2019-11-06 00:23:12
cellbender:remove-background: Running remove-background
cellbender:remove-background: Loading data from directory ./rawdata/filtered_feature_bc_matrix/folder
cellbender:remove-background: CellRanger v3 format
cellbender:remove-background: Trimming dataset for inference.
cellbender:remove-background: Prior on counts in empty droplets is 6001
cellbender:remove-background: Prior on counts for cells is 49252
cellbender:remove-background: Excluding barcodes with counts below 4800
Traceback (most recent call last):
  File "/path/to/software/miniconda3/envs/mypyro/bin/cellbender", line 11, in <module>
    load_entry_point('cellbender', 'console_scripts', 'cellbender')()
  File "~/CellBender/cellbender/base_cli.py", line 101, in main
    cli_dict[args.tool].run(args)
  File "~/CellBender/cellbender/remove_background/cli.py", line 92, in run
    main(args)
  File "~/CellBender/cellbender/remove_background/cli.py", line 185, in main
    run_remove_background(args)
  File "~/CellBender/cellbender/remove_background/cli.py", line 143, in run_remove_background
    args.low_count_threshold)
  File "~/CellBender/cellbender/remove_background/data/dataset.py", line 90, in __init__
    gene_blacklist=gene_blacklist)
  File "~/CellBender/cellbender/remove_background/data/dataset.py", line 252, in _trim_dataset_for_analysis
    f"There are no empty droplets with UMI counts over the lower " \
AssertionError: There are no empty droplets with UMI counts over the lower cutoff of 4800.  Some empty droplets are necessary for the analysis.Reduce the --low-count-threshold parameter.

[sjfleming]

@bobermayer Good to know! Yes, I have seen this kind of error happen before in the case where the “filtered” file is used by mistake. I’ve even done it myself… I should probably try to come up with some heuristic check so that a warning is issued if the file looks like it might be “filtered”.

In fact, I will do this, see #99

[bobermayer]

I just ran into the same error until I realized that I had accidentally passed filtered_feature_bc_matrix.h5 instead of raw_feature_bc_matrix.h5 as input file. it looks like the same thing happened here.