Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Weird side effect from `ctas_approach=True`

See original GitHub issue

Hi,

I just would like to mention a weird side effect that I encountered today.

I’m building a dataframe from an Athena table, like this:

df: pd.DataFrame = wr.athena.read_sql_query(sql=query, database=clean_database)

ctas_approach is true by default.

Later on, I want to mutate one of the columns (which contains a datetime):

df["date"] = df["date"] + pd.DateOffset(months=-2)

This should normally be totally valid pandas. But I get:

File "/Users/jpfrancoia/Library/Caches/pypoetry/virtualenvs/glue-jobs-KVdNkci1-py3.6/lib/python3.6/site-packages/pandas/core/ops/common.py", line 64, in new_method
    return method(self, other)
  File "/Users/jpfrancoia/Library/Caches/pypoetry/virtualenvs/glue-jobs-KVdNkci1-py3.6/lib/python3.6/site-packages/pandas/core/ops/__init__.py", line 503, in wrapper
    result = arithmetic_op(lvalues, rvalues, op, str_rep)
  File "/Users/jpfrancoia/Library/Caches/pypoetry/virtualenvs/glue-jobs-KVdNkci1-py3.6/lib/python3.6/site-packages/pandas/core/ops/array_ops.py", line 193, in arithmetic_op
    res_values = dispatch_to_extension_op(op, lvalues, rvalues)
  File "/Users/jpfrancoia/Library/Caches/pypoetry/virtualenvs/glue-jobs-KVdNkci1-py3.6/lib/python3.6/site-packages/pandas/core/ops/dispatch.py", line 125, in dispatch_to_extension_op
    res_values = op(left, right)
  File "/Users/jpfrancoia/Library/Caches/pypoetry/virtualenvs/glue-jobs-KVdNkci1-py3.6/lib/python3.6/site-packages/pandas/core/ops/common.py", line 64, in new_method
    return method(self, other)
  File "/Users/jpfrancoia/Library/Caches/pypoetry/virtualenvs/glue-jobs-KVdNkci1-py3.6/lib/python3.6/site-packages/pandas/core/arrays/datetimelike.py", line 1332, in __add__
    result = self._add_offset(other)
  File "/Users/jpfrancoia/Library/Caches/pypoetry/virtualenvs/glue-jobs-KVdNkci1-py3.6/lib/python3.6/site-packages/pandas/core/arrays/datetimes.py", line 689, in _add_offset
    result = offset.apply_index(values).tz_localize(self.tz)
  File "pandas/_libs/tslibs/offsets.pyx", line 114, in pandas._libs.tslibs.offsets.apply_index_wraps.wrapper
  File "/Users/jpfrancoia/Library/Caches/pypoetry/virtualenvs/glue-jobs-KVdNkci1-py3.6/lib/python3.6/site-packages/pandas/tseries/offsets.py", line 333, in apply_index
    shifted = liboffsets.shift_months(i.asi8, months)
  File "pandas/_libs/tslibs/offsets.pyx", line 745, in pandas._libs.tslibs.offsets.shift_months
  File "stringsource", line 658, in View.MemoryView.memoryview_cwrapper
  File "stringsource", line 349, in View.MemoryView.memoryview.__cinit__
ValueError: buffer source array is read-only

From what I could understand, it’s somehow related to pyarrow and its immutability (I think, but I could be wrong): https://github.com/apache/arrow/issues/4128.

However, simply querying my dataframe like this completely solves the error:

df: pd.DataFrame = wr.athena.read_sql_query(sql=query, database=clean_database, ctas_approach=False)

Is it expected ? (tested with awswrangler 1.6.3).

Issue Analytics

State:
Created 3 years ago
Comments:5 (5 by maintainers)

Top GitHub Comments

1reaction

JPFrancoiacommented, Jul 28, 2020

Sorry for the delay. I confirm that my code now works with ctas_approach=True 😃 Good job

0reactions

igorborgestcommented, Aug 7, 2020

Hi @JPFrancoia !

It is totally off-topic, but we are stating a “Who uses AWS Data Wrangler?” section. So feel free to add yourself if you want 😄 .

Top Results From Across the Web

Acamprosate (Oral Route) Side Effects - Mayo Clinic

Abdominal pain; confusion; constipation; diarrhea; dry mouth ; Discouragement; feeling sad; irritability; lack of appetite; lack or loss of ...

10 of the Strangest Medication Side Effects - GoodRx

Many medications can cause unusual side effects you might not have heard about. These include severe skin reactions, nerve damage, ...

10 Weird Medication Side Effects | Everyday Health

Upset stomachs, muscle aches, dizziness, and headaches are some of the most common reactions patients have to certain drugs. But what if your...

7 Bizarre Drug Side Effects - Live Science

These seven strange side effects can come from commonly prescribed drugs. ... Commonly prescribed drugs can cause very strange side effects.

Possible Side Effects of Motegrity® (prucalopride)

From CIC relief to possible side effects and dosing, here's what to expect ... Motegrity may cause serious side effects, including: unusual changes...