question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pycharm linter flags output of concat as "type", not DataFrame

See original GitHub issue

Minimum display example

df1 = pd.DataFrame({'A': ['A0']}, index=[0])
df2 = pd.DataFrame({'A': ['A4']}, index=[4])
result = pd.concat([df1, df2])

print(df1.loc[4])  # Pycharm doesn't flag
print(result.loc[4])  # Pycharm flags as "Unresolved attribute reference '.loc' for class 'type'"
                      # But runs fine anyway

Problem description

Pycharm’s linter believes pd.concat as returning class “type”, when it normally returns a dataframe.

I suspect that this is due to core.reshape.concat._Concatenator.get_result() using core.dtypes.concat._get_frame_result_type which sometimes returns SparseDataFrame (a type) and sometimes an object (normally a Dataframe?).

Extract from pandas.core.dtypes.concat

def _get_frame_result_type(result, objs):
    """
    return appropriate class of DataFrame-like concat
    if any block is SparseBlock, return SparseDataFrame
    otherwise, return 1st obj
    """
    if any(b.is_sparse for b in result.blocks):
        from pandas.core.sparse.api import SparseDataFrame
        return SparseDataFrame
    else:
        return objs[0]

Expected Output

The MDE should not throw an error in Pycharm.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.2.final.0 python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None

pandas: 0.20.3 pytest: None pip: 9.0.1 setuptools: 28.8.0 Cython: None numpy: 1.13.1 scipy: None xarray: None IPython: None sphinx: None patsy: None dateutil: 2.6.1 pytz: 2017.2 blosc: None bottleneck: None tables: None numexpr: None feather: None matplotlib: None openpyxl: 2.4.8 xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: 4.6.0 html5lib: None sqlalchemy: None pymysql: None psycopg2: None jinja2: None s3fs: None pandas_gbq: None pandas_datareader: None None

I am using Pycharm Community Edition 2017.1.5.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:2
  • Comments:10 (8 by maintainers)

github_iconTop GitHub Comments

5reactions
jebobcommented, Jul 26, 2017

@gfyoung I’ve been suppressing the warnings in my code using type hints instead. foo = pd.concat([bar1, bar2]) # type: pd.DataFrame

3reactions
gfyoungcommented, May 17, 2018
  • The issue is unrelated to your PyCharm version
  • It should be addressed in 0.23.0
Read more comments on GitHub >

github_iconTop Results From Across the Web

Pandas concat lints as returning "type" : PY-25326
Pycharm's linter believes pd.concat as returning class "type", when it normally returns a Dataframe. Expected Output. Pycharm's linter should resolve the second ...
Read more >
Pycharm does not type hint after dataframe concat
Your type hint is wrong. It should be pd.DataFrame , not pd.Dataframe (capital F ). X_all = pd.concat(objs = [X_train, X_test], ...
Read more >
PEP 484 – Type Hints - Python Enhancement Proposals
Essentially, such a type checker acts as a very powerful linter. ... classes (parameterized or not) to access attributes will result in type...
Read more >
How to Check Data Type of a Pandas Series - Finxter
Pandas does not allow mathematical operations on some data types, such as an object (a string in this case) and an Integer (int64)....
Read more >
Python Tutorial - Getting Started with Python and Python Basics
Python associates types with the objects, not the variables, i.e., a variable can hold ... for -loop, while -loop, input/output, string, list and...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found