SpikeGLXRecordingExtractor fails when gate > 0

Hi @samuelgarcia and all, Initializing a spikeinterface.extractors.SpikeGLXRecordingExtractor object from a probe directory with gate index 0 works properly:

$ ls subject_dir/run_name_g0/run_name_g0_imec0
1-6-2022_g0_t0.imec0.lf.bin   1-6-2022_g0_t1.imec0.lf.bin   1-6-2022_g0_t2.imec0.lf.bin
1-6-2022_g0_t0.imec0.lf.meta  1-6-2022_g0_t1.imec0.lf.meta  1-6-2022_g0_t2.imec0.lf.meta

se.SpikeGLXRecordingExtractor("subject_dir/run_name_g0/run_name_g0_imec0", "imec0.lf") # Assuming folder-per-probe organisation

SpikeGLXRecordingExtractor: 384 channels - 1 segments - 2.5kHz - 7200.001s

… But it fails when the gate index is >0

$ ls subject_dir/run_name_g1/run_name_g1_imec0
1-6-2022_g1_t0.imec0.lf.bin   1-6-2022_g1_t2.imec0.lf.meta  1-6-2022_g1_t5.imec0.lf.bin   1-6-2022_g1_t7.imec0.lf.meta
1-6-2022_g1_t0.imec0.lf.meta  1-6-2022_g1_t3.imec0.lf.bin   1-6-2022_g1_t5.imec0.lf.meta  1-6-2022_g1_t8.imec0.lf.bin
1-6-2022_g1_t1.imec0.lf.bin   1-6-2022_g1_t3.imec0.lf.meta  1-6-2022_g1_t6.imec0.lf.bin   1-6-2022_g1_t8.imec0.lf.meta
1-6-2022_g1_t1.imec0.lf.meta  1-6-2022_g1_t4.imec0.lf.bin   1-6-2022_g1_t6.imec0.lf.meta  1-6-2022_g1_t9.imec0.lf.bin
1-6-2022_g1_t2.imec0.lf.bin   1-6-2022_g1_t4.imec0.lf.meta  1-6-2022_g1_t7.imec0.lf.bin   1-6-2022_g1_t9.imec0.lf.meta

se.SpikeGLXRecordingExtractor("subject_dir/run_name_g1/run_name_g1_imec0", "imec0.lf")

KeyError                                  Traceback (most recent call last)
Input In [47], in <cell line: 1>()
----> 1 se.SpikeGLXRecordingExtractor('.', 'imec0.lf')

File ~/projects/spikeinterface/spikeinterface/extractors/neoextractors/spikeglx.py:38, in SpikeGLXRecordingExtractor.__init__(self, folder_path, stream_id)
     36 if HAS_NEO_10_2:
     37     neo_kwargs['load_sync_channel'] = False
---> 38 NeoBaseRecordingExtractor.__init__(self, stream_id=stream_id, **neo_kwargs)
     40 #~ # open the corresponding stream probe
     41 if HAS_NEO_10_2 and '.ap' in self.stream_id:

File ~/projects/spikeinterface/spikeinterface/extractors/neoextractors/neobaseextractor.py:26, in NeoBaseRecordingExtractor.__init__(self, stream_id, **neo_kwargs)
     24 def __init__(self, stream_id=None, **neo_kwargs):
---> 26     _NeoBaseExtractor.__init__(self, **neo_kwargs)
     28     # check channel
     29     # TODO propose a meachanisim to select the appropriate channel groups
     30     # in neo one channel group have the same dtype/sampling_rate/group_id
     31     # ~ channel_indexes_list = self.neo_reader.get_group_signal_channel_indexes()
     32     stream_channels = self.neo_reader.header['signal_streams']

File ~/projects/spikeinterface/spikeinterface/extractors/neoextractors/neobaseextractor.py:16, in _NeoBaseExtractor.__init__(self, **neo_kwargs)
     14 neoIOclass = eval('neo.rawio.' + self.NeoRawIOClass)
     15 self.neo_reader = neoIOclass(**neo_kwargs)
---> 16 self.neo_reader.parse_header()
     18 assert self.neo_reader.block_count() == 1, \
     19     'This file is neo multi block spikeinterface support one block only dataset'

File ~/miniconda3/envs/rebooting/lib/python3.10/site-packages/neo/rawio/baserawio.py:185, in BaseRawIO.parse_header(self)
    172 def parse_header(self):
    173     """
    174     This must parse the file header to get all stuff for fast use later on.
    175
   (...)
    183
    184     """
--> 185     self._parse_header()
    186     self._check_stream_signal_channel_characteristics()

File ~/miniconda3/envs/rebooting/lib/python3.10/site-packages/neo/rawio/spikeglxrawio.py:106, in SpikeGLXRawIO._parse_header(self)
    103 signal_channels = []
    104 for stream_name in stream_names:
    105     # take first segment
--> 106     info = self.signals_info_dict[0, stream_name]
    108     stream_id = stream_name
    109     stream_index = stream_names.index(info['stream_name'])

KeyError: (0, 'imec0.lf')

Same issue when instantiating directly the SpikeGLXRawIO object from neo. It seems like the gate (and not only the stream) should be either passed as a kwarg or parsed from the folder name?

Thanks!

Spikeinterface version = 0.94.1.dev0 neo version = 0.12.2

[samuelgarcia]

@grahamfindlay @billkarsh : I propose to move the discussion where I did the code here : https://github.com/NeuralEnsemble/python-neo/pull/1125

[samuelgarcia]

Hi Graham. Thanks a lot for this. I will push myself to gin repo. This look absolutly perfect. bin files are small and I think we cover many case.