question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

_metadata items of subclassed pd.Series are not propagated into corresponding SubclassedDataFrame

See original GitHub issue

Code Sample

import pandas as pd

# Define a subclass for pd.Series with metadata myprop
class SubclassedSeries(pd.Series):

    _metadata = ['myprop']

    @property
    def _constructor(self):
        return SubclassedSeries

    @property
    def _constructor_expanddim(self):
        return SubclassedDataFrame

# Define a subclass for pd.DataFrame that slices to SubclassedSeries
class SubclassedDataFrame(pd.DataFrame):
    @property
    def _constructor(self):
        return SubclassedDataFrame

    @property
    def _constructor_sliced(self):
        return SubclassedSeries

# make an instance of SubclassedSeries and set myprop
sr = SubclassedSeries([1,2,3])
sr.myprop = 'foo'
print("myprob is", sr.myprop) # Works

# put the SubclassedSeries object into a SubclassedDataFrame and try to get myprop
df = SubclassedDataFrame({'a': sr})
print(type(df.a)) # Works (prints <class '__main__.SubclassedSeries'>)
print("myprob is", df.a.myprop) # does not work (AttributeError: 'SubclassedSeries' object has no attribute 'myprop')

Problem description

_metadata items of pd.Series subclasses are not propagated when the SubclassedSeries object is put into a SubclassedDataFrame. I would expect myprop to be available in the new SubclassedDataFrame.

Expected Output

myprob is foo
<class '__main__.SubclassedSeries'>
myprob is foo

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : None python : 3.8.1.final.0 python-bits : 64 OS : Linux OS-release : 5.3.0-42-lowlatency machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : de_DE.UTF-8 LOCALE : de_DE.UTF-8

pandas : 1.0.2 numpy : 1.18.1 pytz : 2019.3 dateutil : 2.8.1 pip : 20.0.2 setuptools : 46.0.0.post20200309 Cython : 0.29.15 pytest : 5.4.1 hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : 2.11.1 IPython : 7.13.0 pandas_datareader: None bs4 : None bottleneck : None fastparquet : None gcsfs : None lxml.etree : None matplotlib : 3.1.3 numexpr : 2.7.1 odfpy : None openpyxl : None pandas_gbq : None pyarrow : None pytables : None pytest : 5.4.1 pyxlsb : None s3fs : None scipy : 1.4.1 sqlalchemy : None tables : 3.6.1 tabulate : None xarray : 0.15.0 xlrd : 1.2.0 xlwt : None xlsxwriter : None numba : None

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:9 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
samuelduchesnecommented, Sep 22, 2020

@Flix6x Somehow, returning a function instead of a class fails at line 396: https://github.com/pandas-dev/pandas/blob/8f6ec1e83a5de6d30cd187b832853797fd569d31/pandas/core/reshape/concat.py#L394-L398 with error AttributeError: 'function' object has no attribute '_get_axis_number'

Any workarounds?

0reactions
jorisvandenbosschecommented, Feb 16, 2022

I opened a PR to fix this _get_axis_number case: https://github.com/pandas-dev/pandas/pull/46018

Read more comments on GitHub >

github_iconTop Results From Across the Web

subclassing pandas Series and DataFrame: When combining ...
I get an AttributeError: 'MyFrame' object has no attribute '_md' as the metadata is not propagated. Is this a normal behavior or am...
Read more >
Extending pandas — pandas 1.5.2 documentation
Below example shows how to define SubclassedSeries and SubclassedDataFrame overriding constructor properties. class SubclassedSeries(pd.Series): @property ...
Read more >
dask.dataframe.Series.map - Dask documentation
If 'ignore', propagate NaN values, without passing them to the mapping correspondence. metapd.DataFrame, pd.Series, dict, iterable, tuple, optional. An empty pd ...
Read more >
Pandas Series: map() function - w3resource
Mapping correspondence. function, dict, or Series, Required. na_action, If 'ignore', propagate NaN values, without passing them to the mapping ...
Read more >
Using Pandas and Python to Explore Your Dataset
Getting to Know Pandas' Data Structures. Understanding Series Objects; Understanding DataFrame Objects. Accessing Series Elements.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found