Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DataFrame viewer crashes on a DF with invalid data

See original GitHub issue

DF looks like this

Not what I wanted it to look like, but I’m trying to build a df and wanted to see if it worked or not.

However the DF viewer crashes with this:

    126     # Turn into JSON using pandas. We use pandas because it's about 3 orders of magnitude faster to turn into JSON
    127     rows = df.iloc[start:end]
--> 128     rows = rows.replace(
    129         {
    130             _VSCODE_np.inf: "inf",

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\frame.py in replace(self, to_replace, value, inplace, limit, regex, method)
   4374         method="pad",
   4375     ):
-> 4376         return super().replace(
   4377             to_replace=to_replace,
   4378             value=value,

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\generic.py in replace(self, to_replace, value, inplace, limit, regex, method)
   6489                 to_replace, value = keys, values
   6490 
-> 6491             return self.replace(
   6492                 to_replace, value, inplace=inplace, limit=limit, regex=regex
   6493             )

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\frame.py in replace(self, to_replace, value, inplace, limit, regex, method)
   4374         method="pad",
   4375     ):
-> 4376         return super().replace(
   4377             to_replace=to_replace,
   4378             value=value,

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\generic.py in replace(self, to_replace, value, inplace, limit, regex, method)
   6532                         )
   6533                     self._consolidate_inplace()
-> 6534                     new_data = self._mgr.replace_list(
   6535                         src_list=to_replace,
   6536                         dest_list=value,

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\internals\managers.py in replace_list(self, src_list, dest_list, inplace, regex)
    640         mask = ~isna(values)
    641 
--> 642         masks = [comp(s, mask, regex) for s in src_list]
    643 
    644         result_blocks = []

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\internals\managers.py in <listcomp>(.0)
    640         mask = ~isna(values)
    641 
--> 642         masks = [comp(s, mask, regex) for s in src_list]
    643 
    644         result_blocks = []

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\internals\managers.py in comp(s, mask, regex)
    634 
    635             s = com.maybe_box_datetimelike(s)
--> 636             return _compare_or_regex_search(values, s, regex, mask)
    637 
    638         # Calculate the mask once, prior to the call of comp

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\internals\managers.py in _compare_or_regex_search(a, b, regex, mask)
   2002         result = tmp
   2003 
-> 2004     _check_comparison_types(result, a, b)
   2005     return result
   2006 

~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\internals\managers.py in _check_comparison_types(result, a, b)
   1969                 type_names[0] = f"ndarray(dtype={a.dtype})"
   1970 
-> 1971             raise TypeError(
   1972                 f"Cannot compare types {repr(type_names[0])} and {repr(type_names[1])}"
   1973             )

Issue Analytics

State:
Created 3 years ago
Comments:8 (8 by maintainers)

Top GitHub Comments

1reaction

joyceerhlcommented, Feb 9, 2021

I need to fix this before we release for Feb, this would be a regression

1reaction

rchiodocommented, Feb 8, 2021

And it repros with latest changes.

Top Results From Across the Web

Pandas dataframe read_csv on bad data - Stack Overflow

here is my way to solve those problem, it is slow but works so well, Simply says just read the CSV file as...

pyspark.sql module - Apache Spark

SparkSession Main entry point for DataFrame and SQL functionality. pyspark.sql.DataFrame A distributed collection of data grouped into named columns. pyspark.

Python console crashes when trying to view a pandas v 0.24.1 ...

Python console crashes when trying to view a pandas v 0.24.1 DataFrame with a Int64 column with nans (SciView). 17. Relates to 4...

Pandas read_csv to DataFrames: Python Pandas Tutorial

In this tutorial, we'll show how to use read_csv pandas to import data into Python, with practical examples. csv (comma-separated values) ...

10 minutes to pandas — pandas 1.5.2 documentation

Use DataFrame.head() and DataFrame.tail() to view the top and bottom rows of the frame ... In [21]: df.sort_index(axis=1, ascending=False) Out[21]: ...