Timestamp_16 support

Currently in FIT we have timestamps which are 32bit and timestamp_16 which are 16bit difference from previous timestamp and are currently not read into nice values.

I wrote generator function based on ParseVivosmartHR. Here seems to be offical function how to do it (in Java):

mesgTimestamp += ( timestamp_16 - ( mesgTimestamp & 0xFFFF ) ) & 0xFFFF; 

It works but it definitely isn’t written as it should, but I don’t know fitparse enough to improve it.

#Input is what we get in FitFile.get_messages
def fix_times(messages):                                                        
    timestamp=None                                                              
    timestamp_16=None                                                           
    real_time=0                                                                 
    UTC_REFERENCE = 631065600  # timestamp for UTC 00:00 Dec 31 1989            
    for message in messages:                                                    
        #print (message)                                                        
        #message["full_timestamp"]="TIMESTAMP"                                  
        field = message.get("timestamp")                                        
        if field is not None:                                                   
            timestamp = field.raw_value                                         
            #I'm not sure if this is correct to set this to None. Without it records with just timestamp doesn't have same new timestamp anymore which could be a bug or it should be that way                                          
            timestamp_16 = None                                                 
        tm_16 = message.get("timestamp_16")                                     
        if tm_16 is not None:                                                   
            timestamp_16 = tm_16.raw_value                                      
        #print(timestamp, timestamp_16)                                         
        if timestamp_16 is None:                                                
            ts_value = timestamp                                                
        else:                                                                   
            new_time = int( timestamp/2**16) * 2**16 + timestamp_16             
            if new_time >= real_time:                                           
                real_time = new_time                                            
            else:                                                               
                real_time = new_time + 2**16                                    
            ts_value = real_time                                                
        if ts_value is not None and ts_value >= 0x10000000:                     
            value = datetime.datetime.utcfromtimestamp(UTC_REFERENCE            
                    + ts_value)                                                 
        else:                                                                   
            value = None                                                        
        message.fields.append(                                                  
                FieldData(                                                      
                    field_def=None,                                             
                    field=FIELD_TYPE_TIMESTAMP_1,                               
                    parent_field=None,                                          
                    value=value,                                                
                    raw_value=ts_value                                          
                    ))                                                          
        yield message

[tuxinaut]

@francescolosterzo really cool work 👍 and yes I’m pretty sure that’s timezone related (take a look on the code below)

>>> timestamp_16 = 31132
>>> tz = pytz.timezone('UTC')
>>> utc_offset = int(datetime.datetime.timestamp(datetime.datetime(1989,12,31,tzinfo=tz)))
>>> timestamp = int(datetime.datetime.timestamp(datetime.datetime(2019,4,30,22,0,tzinfo=tz)))
>>> timestamp_raw = timestamp - utc_offset
>>> out = timestamp_raw
>>> out += ( timestamp_16 - ( out & 0xFFFF ) ) & 0xFFFF
>>> datetime.datetime.fromtimestamp(out + utc_offset,tz=tz)
datetime.datetime(2019, 4, 30, 22, 1, tzinfo=<UTC>)

[francescolosterzo]

thanks @tuxinaut ! I tried it but I still get a sizeable difference, i.e.: datetime.datetime(2019, 4, 30, 18, 36, 44)

On the other hand I browsed the details of the code and I see that the UTC offset shift due to Garmin’s different starting time is applied to timestamp, but I cannot find anything similar for timestamp_16.

Following this hypothesis I tried to work with raw timestamp* values (I guess they are the values before this UTC offset correction) and here is what I get:

# values taken from full printouts
timestamp_raw = 925596000
timestamp_16_raw = 31132

out = timestamp_raw
out += ( timestamp_16 - ( out & 0xFFFF ) ) & 0xFFFF

datetime.datetime.fromtimestamp(out + utc_offset)

The result is datetime.datetime(2019, 4, 30, 23, 1)

[here is the full notebook, just search for “925596000”]

This is the closest I got so far to the right result, and it is shifted only by one hour (the minute might be due to the granularity of the data) which might now be a timezone related issue.

What do you think @dtcooper ?