using the correct marker timestamp from matlab to avoid an accumulating delay

tl;dr: How can I use lsl_local_clock to send precise markers from matlab to muselsl via outlet.push_sample(marker,timestamp)?

Problem: When sending markers in matlab using outlet.push_sample(marker, timestamp), using timestamp of 0 or leaving blank does not use the current lsl_local_clock (despite what documentation suggests). It actually will just send the flag to the very first sample in the dataset. So i’m forced to either:

1. Use a high precision time from matlab/psychtoolbox. This works, but the clocks between matlab and muselsl seem to accumulate a delay between them as a function of muselsl stream runtime duration. See attached figure to show how the delay between GetSecs in matlab and the final timestamp from [~, lsltime] = inlet.pull_chunk grows as a function of time (x-axis = sample #, y-axis = difference between GetSecs and lsltime.). I collected this over a 2 hour period, sampling GetSecs and lsltime every 1 second. zoomed in to see each sample over a 5 minute time period:

2. Before sending a marker from matlab, use inlet.pull_chunk and use the latest timestamp to send the flag. This avoids the accumulating delay, but is too jittered. I can’t use pull_sample because that just pulls the next sample available in the stream, not the most recent sample.

apparatus:

• ubuntu 16.04
• muselsl v1.0.0
• matlab 2018a with Psychtoolbox 3.0.15 and labstreaminglayer
• muse 2016

setup to collect data for the above figures to show the accumulating delay… open muselsl stream open muselsl record in matlab, ran:

lib = lsl_loadlib();
info = lsl_streaminfo(lib,'Marker','Markers',3,0,'cf_string');
outlet = lsl_outlet(info);

disp('Resolving an EEG stream...');
result = {};
while isempty(result)
    result = lsl_resolve_byprop(lib,'type','EEG');
end

% create a new inlet
disp('Opening an inlet...');
inlet = lsl_inlet(result{1}); % create inlet for getting proper timestamps

% collect data for 2 hours, sending a flag every 1 second... (change 7200secs / 60 = 120 mins)
tEnd=GetSecs;
for ii = 1:7200
    
    fprintf('Second %i of 7200... \n', ii);
    
    while(GetSecs-tEnd) < 1
    end
    tEnd = GetSecs;
    
    [~, lsltime] = inlet.pull_chunk;
    tcor=GetSecs-tEnd;
    if ~isempty(lsltime)
        lsltime = lsltime(end);
    else
        [~, lsltime] = inlet.pull_sample;
        tcor=GetSecs-tEnd;
    end
    outlet.push_sample({'flag', num2str(tEnd), num2str(tcor)}, lsltime-tcor)
    
end

The accumulating delay is slow and is linear. We discovered this delay when collecting actual data for an ERP study. Here is the single-trial ERP at time 0 and again, for the same participant and experiment, 2 hours later. The delay accumulates slow enough that you can get an ERP in a 5 minute task, but you cannot reasonably combined trials collected with a significant delay (scale of several minutes) between them. In these plots, the ERP is time-locked to the time of stimulus presentation (obtained via high-precision VBLTimestamp of the output of Screen(‘Flip’) in matlab/psychtoolbox). You can even see the accumulation of the delay in the single-trial ERPs… I can time-lock these to the LSLtime obtained via inlet.pull_chunk, and it sorta works, but it’s way too jittered to be a good solution.

Some other things that might be worth noting: I’ve tried turning the dejitter in muselsl off, and it doesnt seem to make any difference. The delay is entirely dependent on when the muselsl stream has started…one temporary solution is to restart the stream every few minutes to minimuze the effects of the delay, but that’s a really bad solution. The delay is seen whether muselsl and matlab are on the same or different computers. I only have access to linux computers, so maybe it’s OS dependent? It is only on linux that the system clock that LSL and Psychtoolbox/matlab use are the same.

[HashemiScience]

@alexandrebarachant your adaptive timestamp (#101 ) works perfectly. I just collected data on the same experiment (3 minutes of data collection), 2 hours apart, and got these beautiful ERPs without any sign of the drift.

THANK YOU!

for the record, I am sending my markers using the high precision time of the local computer (GetSecs in matlab).

Time 1:

Two hours later:

[alexandrebarachant]

Okay, i made a PR that should fix the issue by correcting the timestamps directly in the muse base class.

Looking more closely at the matlab code you provided, i think the issues come from the fact that you are trying to apply a correction to your marker timestamps before sending them.

if you are using muselsl to record you data, you can just send the markers with the timestamp corresponding to you local machine time, and let the recorder do the correction i.e. no need to pull emg sample in matlab to get their timestamp.