Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Verify CXG conversion works for certain datasets

See original GitHub issue

Link to Collection with errored dataset: https://cellxgene.cziscience.com/collections/60358420-6055-411d-ba4f-e8ac80682a2e/private

Logs link: https://us-west-2.console.aws.amazon.com/batch/v2/home?region=us-west-2#jobs/detail/8e7ff995-3b20-48e5-83a1-8247873ff414

Stack trace:

File "/usr/local/lib/python3.8/dist-packages/backend/czi_hosted/common/utils/cxg_generation_utils.py", line 75, in convert_dataframe_to_cxg_array
array[:] = value\n  File "tiledb/libtiledb.pyx", line 4885, in tiledb.libtiledb.DenseArrayImpl.__setitem__
File "tiledb/libtiledb.pyx", line 415, in tiledb.libtiledb._write_array
File "tiledb/libtiledb.pyx", line 479, in tiledb.libtiledb._raise_ctx_err
File "tiledb/libtiledb.pyx", line 464, in tiledb.libtiledb._raise_tiledb_error
tiledb.libtiledb.TileDBError: [TileDB::IO] Error: Cannot open file \'/local.cxg/obs/__1617759016805_1617759016805_a11917d04be346c2a937ba82cbd16ae8_6/BEST4/OTOP2 Cells.tdb\'; No such file or directory

TEAM COMMENTS:

Old tileDB version didn’t process slashes correctly.
tileDB just put out 0.19, which fixes this issue. Other tickets in this epic upgrade the version, this ticket is to verify that it worked for this dataset.
In scope: add a test with a slash in the observation, make sure it uploads successfully.

Issue Analytics

State:
Created 2 years ago
Comments:10 (3 by maintainers)

Top GitHub Comments

1reaction

nguyenvcommented, Sep 22, 2021

Hi @metakuni, I was able to reproduce your findings. It looks like this was in fact initially supported by this PR in TileDB Embedded 2.2.

1reaction

metakunicommented, Sep 22, 2021

Finally managed to reproduce the error using repro test script from https://github.com/TileDB-Inc/TileDB-Py/issues/294.

The test seems to fail only for tiledb versions <0.8.1. tiledb 0.8.1+ appears to work fine, so perhaps the fixed version quoted in the bug comment was incorrect.

Using tiledb==0.8.0:

 $ python badattrname.py 
tiledb.__version__: 0.8.0
tiledb.libtiledb.version(): (2, 2, 2)
os.name: posix
os.uname(): posix.uname_result(sysname='Darwin', nodename='czipa1osx591', release='20.6.0', version='Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64', machine='x86_64')
sys.platform: darwin
...
Traceback (most recent call last):
  File "badattrname.py", line 24, in <module>
    A[:] = value
  File "tiledb/libtiledb.pyx", line 4909, in tiledb.libtiledb.DenseArrayImpl.__setitem__
  File "tiledb/libtiledb.pyx", line 416, in tiledb.libtiledb._write_array
  File "tiledb/libtiledb.pyx", line 480, in tiledb.libtiledb._raise_ctx_err
  File "tiledb/libtiledb.pyx", line 465, in tiledb.libtiledb._raise_tiledb_error
tiledb.libtiledb.TileDBError: [TileDB::IO] Error: Cannot open file '/Users/kunikatsuya/PycharmProjects/single-cell-data-portal/foo/__1632322386089_1632322386089_2ce334ad46324464bb0478815513e1a2_7/fo/o.tdb'; No such file or directory