Different behaviour on two different environments. TypeError: data type "datetime" not understood

I have an aggregation logic

In [19]: datewise_prices
Out[19]: 
[{'arrivalDate': 1490227200000,
  'maxPrice': 3300,
  'minPrice': 3300,
  'modalPrice': 3300},
 {'arrivalDate': 1490400000000,
  'maxPrice': 3300,
  'minPrice': 3300,
  'modalPrice': 3300},
 {'arrivalDate': 1490832000000,
  'maxPrice': 3500,
  'minPrice': 3500,
  'modalPrice': 3500},
 {'arrivalDate': 1490918400000,
  'maxPrice': 3300,
  'minPrice': 3300,
  'modalPrice': 3300},
 {'arrivalDate': 1491091200000,
  'maxPrice': 3300,
  'minPrice': 3300,
  'modalPrice': 3300}]

In [20]: weekly_dataframe = pandas.DataFrame(datewise_prices)
    ...: weekly_dataframe.drop('minPrice', axis=1, inplace=True)
    ...: weekly_dataframe.drop('maxPrice', axis=1, inplace=True)
    ...: weekly_dataframe['arrivalDate'] = pandas.to_datetime(weekly_dataframe['arrivalDate'], unit='ms')
    ...: weekly_dataframe = weekly_dataframe.resample('W', on='arrivalDate')['modalPrice'].mean().dropna().reset_index()
    ...: weekly_dataframe['label'] = (weekly_dataframe['arrivalDate'] - pandas.offsets.DateOffset(days=6)).dt.strftime('%d %b') + ' to ' + weekly_dataframe['arriv
    ...: alDate'].dt.strftime('%d %b')

It results in an error on my Staging environment but works fine on my local.

The staging environment gives an error in (weekly_dataframe['arrivalDate'] - pandas.offsets.DateOffset(days=6)).dt.strftime('%d %b')

Traceback (most recent call last):
  File "/root/myapp/myapp/handlers/aggregation_handler.py", line 290, in get_daily_weekly_and_monthly_aggregates
    weekly_dataframe['label'] = (weekly_dataframe['arrivalDate'] - pandas.offsets.DateOffset(days=6)).dt.strftime('%d %b') + ' to ' + weekly_dataframe['arrivalDate'].dt.strftime('%d %b')
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/ops.py", line 696, in wrapper
    converted = _Op.get_op(left, right, name, na_op)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/ops.py", line 330, in get_op
    return _TimeOp(left, right, name, na_op)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/ops.py", line 343, in __init__
    lvalues = self._convert_to_array(left, name=name)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/ops.py", line 454, in _convert_to_array
    if (inferred_type in ('datetime64', 'datetime', 'date', 'time') or
TypeError: data type "datetime" not understood

but it works fine on my local.

Output of pd.show_versions() on local

INSTALLED VERSIONS ------------------ commit: None python: 2.7.6.final.0 python-bits: 64 OS: Linux OS-release: 4.4.0-78-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_IN LOCALE: None.None

pandas: 0.19.2 nose: None pip: 9.0.1 setuptools: 36.0.1 Cython: None numpy: 1.12.1 scipy: None statsmodels: None xarray: None IPython: 5.4.1 sphinx: None patsy: None dateutil: 2.5.3 pytz: 2016.7 blosc: None bottleneck: None tables: None numexpr: None matplotlib: None openpyxl: None xlrd: None xlwt: None xlsxwriter: 0.9.6 lxml: None bs4: None html5lib: 0.999 httplib2: 0.8 apiclient: None sqlalchemy: None pymysql: None psycopg2: None jinja2: 2.8 boto: 2.40.0 pandas_datareader: None

Output of pd.show_versions() on Staging

INSTALLED VERSIONS ------------------ commit: None python: 2.7.6.final.0 python-bits: 64 OS: Linux OS-release: 3.13.0-57-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.None

pandas: 0.19.2 nose: None pip: 1.5.4 setuptools: 3.3 Cython: None numpy: 1.13.0 scipy: None statsmodels: None xarray: None IPython: 4.0.0 sphinx: None patsy: None dateutil: 2.5.3 pytz: 2017.2 blosc: None bottleneck: None tables: None numexpr: None matplotlib: None openpyxl: None xlrd: None xlwt: None xlsxwriter: 0.9.6 lxml: None bs4: None html5lib: 0.999 httplib2: None apiclient: None sqlalchemy: 1.0.8 pymysql: None psycopg2: None jinja2: 2.9.6 boto: 2.40.0 pandas_datareader: None

Please help me out.

[hussaintamboli]

Just figured it out myself.

It’s because of numpy==1.13.0. For numpy==1.12.1, it’s working fine.

[hussaintamboli]

@jorisvandenbossche Please check