`to_featureset` does not preserve datetime

Describe the bug When attempting to append data by converting features in a spatially enabled dataframe to a featureset (using to_featureset()) and then edit_features to update an AGOL layer, datetime attrributes are converted to unix timestamps. If an AGOL layer contains date columns, then that data is not appended to the hosted feature service.

The code below is connected to another github issue.

To Reproduce Steps to reproduce the behavior:

## pull layer to update
item_id = ## item_id
to_update = gis.content.get(itemid=item_id) 
to_update_layer = to_update.layers[0]

## add new data
format = "%Y-%m-%d %H:%M:%S"

df = pd.DataFrame([
    {'test' : 'update4',
     'longitude' : -117.15246191281217,
     'latitude' : 32.70969794547845,
     'date' : today, #str(datetime.now().strftime(format))
    },
     {'test' : 'update5',
     'longitude' : -115.15246191281217,
     'latitude' : 32.70969794547845,
     'date' : today, #str(datetime.now().strftime(format))
     },
     {'test' : 'update6',
     'longitude' : -117.15246191281217,
     'latitude' : 34.70969794547845,
     'date' : today, #str(datetime.now().strftime(format))
     },
    ])


## create timezone cols
df['date_datetime'] = pd.to_datetime(df['date'])
df['date_localized_utc'] = df['date_datetime'].dt.tz_localize('UTC')
df['date_localized_est'] = df['date_localized_utc'].dt.tz_convert('US/Eastern')
df['date_localized_pst'] = df['date_localized_utc'].dt.tz_convert('US/Pacific')
df['date_datetime_epoch'] = df['date_datetime'].map(pd.Timestamp.timestamp)

update_with = pd.DataFrame.spatial.from_xy(df=df, x_column='longitude', y_column='latitude', sr=4326)
print(update_with.dtypes)
print(list(update_with.dropna(how='all', axis=1).spatial.to_featureset()))

## update data
results = []
for i, row in update_with.iterrows():
    try: 
        featureset = update_with.iloc[i:i+1].dropna(how='all', axis=1).spatial.to_featureset()
        result = to_update.layers[0].edit_features(adds=featureset)
        results.append(result)
    except:
        print(f'Feature {i} not added')
results

Screenshots

Expected behavior Datetime columns are maintained as dates and appended to AGOL successfully (this just may not be possible via to_featureset / edit_features; if so, please let me know!😄).

Platform (please complete the following information):

• OS: Windows
• Browser Chrome/Firefox
• Python API Version 2.0.0

[Ray-Mangan]

For the column truncation I am not sure why Shapefile does that. If I find anything out about it or a way around I will let you know.

@nanaeaubry - I know this issue is closed but the column truncation issue is caused by the shapefile format. The format uses .dbf as the datatable storage format and has a 10 character limit. See the limitations section part way down the page here: https://en.wikipedia.org/wiki/Shapefile

[nanaeaubry]

Great to hear that it works! Creating a shapefile occurs when arcpy is not in your environment. I assumed it was which is why I spoke about fgdb but if it is not then we use the Shapely library and create a shapefile instead so there is no workflow difference just the result is a Feature Layer and a Shapefile instead.

For the column truncation I am not sure why Shapefile does that. If I find anything out about it or a way around I will let you know.

I will go ahead and close this issue since we were able to resolve one problem and the other is linked to the other issue you have open. You can still post here if needed but for timezone updates they will be on your other issue 😃 Thanks!