Not found cell and umi barcode in entry xx of the bam file

I am testing amazon AWS to run Velocyto by using 10X PBMC_10K.bam file. Velocyte (one step without splitting a samtools step) seems to run fine without an error (although it takes greater than 7hours!!) however, it will not complete the run if a sorted bam file is created first separately by samtools as recommenced in the velocity cli guide.

Anyway, I created a sorted bam file by

samtools sort -l 7 -m 2048M -t CB -O BAM -@100 -o /temp/home/cellsorted_PBMC.bam /temp/home/PBMC_10K.bam

and fired up Velocyte by

velocyto run -b filtered_feature_bc_matrix/barcodes.tsv -o /temp/home -m GRCh38_rmsk.gtf cellsorted_PBMC.bam refdata-gex-GRCh38-2020-A/genes/genes.gtf.

Then I got the error:

2020-09-27 21:17:36,428 - INFO - No SAMPLEID specified, the sample will be called cellsorted_PBMC_CQIPR (last 5 digits are a random-id to avoid overwriting some other file by mistake)
2020-09-27 21:17:36,428 - DEBUG - Using logic: Default
2020-09-27 21:17:36,438 - INFO - Read 10985 cell barcodes from /temp/home/filtered_feature_bc_matrix/barcodes.tsv
2020-09-27 21:17:36,438 - DEBUG - Example of barcode: AAACCCAGTATATGGA and cell_id: cellsorted_PBMC_CQIPR:AAACCCAGTATATGGA-1
2020-09-27 21:17:36,448 - DEBUG - Peeking into /temp/home/cellsorted_PBMC.bam
2020-09-27 21:17:36,449 - WARNING - Not found cell and umi barcode in entry 0 of the bam file
.......

E::idx_find_and_load] Could not retrieve index file for '/temp/home/cellsorted_PBMC.bam'
Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/velocyto", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/.local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/.local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/velocyto/commands/run.py", line 113, in run
    return _run(bamfile=bamfile, gtffile=gtffile, bcfile=bcfile, outputfolder=outputfolder,
  File "/home/ubuntu/.local/lib/python3.8/site-packages/velocyto/commands/_run.py", line 159, in _run
    exincounter.peek(bamfile[0])
  File "/home/ubuntu/.local/lib/python3.8/site-packages/velocyto/counter.py", line 158, in peek
    raise IOError("The bam file does not contain cell and umi barcodes appropriatelly formatted. If you are runnin UMI-less data you should use the -U flag.")
OSError: The bam file does not contain cell and umi barcodes appropriatelly formatted. If you are runnin UMI-less data you should use the -U flag.

AWS set is as follows;

cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Also, some cpu info;

~ lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   48 bits physical, 48 bits virtual
CPU(s):                          64
On-line CPU(s) list:             0-63
Thread(s) per core:              2
Core(s) per socket:              32
Socket(s):                       1
NUMA node(s):                    4
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           1
Model name:                      AMD EPYC 7571
Stepping:                        2
CPU MHz:                         2477.215
BogoMIPS:                        4399.39
Hypervisor vendor:               KVM
Virtualization type:             full
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdp
                                 e1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma c
                                 x16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefet
                                 ch topoext perfctr_core vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 clzero xsave
                                 erptr arat npt nrip_save

I really appreciate any inputs to solve this issue.

Thanks.

[akhst7]

@kaizen89, You can just let Velocity run till the end, as long as there is not issue. Apparently, my issue raises when the samtool sort step is performed separately form Velocyto. As long you let Velocyto run samtools sort , it will be ok to run the whole process.

[akhst7]

@Junedays

If sam is not sorted internally or externally, Velocyto will complain and stop. You can run either top or htop and check Velocyte output to see what Velocyte is doing live. Velocyte runs samtools sort first if bam is not sorted and then proceed to the alignment.