Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

cutadapt installed via conda igzip error for some fastq files

See original GitHub issue

Only very recently (~2 weeks ago), cutadapt installed via conda has the following error:

This is cutadapt 3.2 with Python 3.8.6
Command line parameters: -j 4 -e 0.1 -q 20 -O 1 -a AGATCGGAAGAGC in2438_3_CKDL210000739-2a-AK5142-AK6697_HVHF2DSXY_L2_1.fq.gz
Processing reads on 4 cores in single-end mode ...
[--------->8 ] 00:00:26     5,536,084 reads  @      4.9 µs/read;  12.33 M reads/minuteERROR: Traceback (most recent call last):
  File "/opt/conda/envs/test/lib/python3.8/site-packages/cutadapt/pipeline.py", line 560, in run
    self.send_to_worker(chunk_index, chunk)
  File "/opt/conda/envs/test/lib/python3.8/site-packages/xopen/__init__.py", line 141, in __exit__
    self.close()
  File "/opt/conda/envs/test/lib/python3.8/site-packages/xopen/__init__.py", line 327, in close
    self._raise_if_error(allow_sigterm=allow_sigterm)
  File "/opt/conda/envs/test/lib/python3.8/site-packages/xopen/__init__.py", line 350, in _raise_if_error
    raise OSError("{} (exit code {})".format(message, retcode))
OSError: igzip: Error while decompressing extra concatenatedgzip files on in2438_3_CKDL210000739-2a-AK5142-AK6697_HVHF2DSXY_L2_1.fq.gz (exit code 1)

ERROR: Traceback (most recent call last):
  File "/opt/conda/envs/test/lib/python3.8/site-packages/cutadapt/pipeline.py", line 626, in run
    raise e
OSError: igzip: Error while decompressing extra concatenatedgzip files on in2438_3_CKDL210000739-2a-AK5142-AK6697_HVHF2DSXY_L2_1.fq.gz (exit code 1)

Traceback (most recent call last):
  File "/opt/conda/envs/test/bin/cutadapt", line 10, in <module>
    sys.exit(main_cli())
  File "/opt/conda/envs/test/lib/python3.8/site-packages/cutadapt/__main__.py", line 845, in main_cli
    main(sys.argv[1:])
  File "/opt/conda/envs/test/lib/python3.8/site-packages/cutadapt/__main__.py", line 912, in main
    stats = r.run()
  File "/opt/conda/envs/test/lib/python3.8/site-packages/cutadapt/pipeline.py", line 824, in run
    raise e
OSError: igzip: Error while decompressing extra concatenatedgzip files on in2438_3_CKDL210000739-2a-AK5142-AK6697_HVHF2DSXY_L2_1.fq.gz (exit code 1)

This only happens with some fastq files, only in multi-thread mode (with -j specified), and only with conda-installed cutadapt. I’ve tried version 3.2 and 3.1. I’ve tried rolling back versions of some dependencies including pigz=2.3.4, xopen=1.0.1, dnaio=0.4.4, none of these helps. I understand this is probably a bioconda problem rather than cutadapt problem. This has also been reported in bioconda recipes repo. So far no solutions. I am hoping maybe cutadapt developers might point me to a few places since you understand the error message better. Thank you.