question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Unexpected output of `to_json` for `complex` data

See original GitHub issue
  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd
s = pd.Series({0: 0+2j, 1: 3+4j, 2: 5+6j})
s.to_json()

Problem description

Calling to_json() on a DataFrame or Series with complex data gives unexpected results. Running the above Code Sample gives the following output:

'{"0":{"imag":2.0},"1":{"imag":4.0},"2":{"imag":6.0}}'

This is a problem both because (1) the output is not terribly useful (the real part is just dropped), and (2) according to the docs this should raise an error (see the following snippet copied from the linked docs page):

>>> DataFrame([1.0, 2.0, complex(1.0, 2.0)]).to_json()  # raises
RuntimeError: Unhandled numpy dtype 15

Expected Output

According to the docs, this should at least raise a RuntimeError. However, it would be nice for this to actually be supported.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 0158382ac8459a197d088790fd45f433c83c25c8 python : 3.8.5.final.0 python-bits : 64 OS : Darwin OS-release : 20.1.0 Version : Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:11 PDT 2020; root:xnu-7195.50.7~2/RELEASE_X86_64 machine : x86_64 processor : i386 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8

pandas : 1.3.0.dev0+1441.g0158382ac8 numpy : 1.20.2 pytz : 2021.1 dateutil : 2.8.1 pip : 20.1.1 setuptools : 47.1.0 Cython : 0.29.23 pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : None IPython : 7.22.0 pandas_datareader: None bs4 : None bottleneck : None fsspec : None fastparquet : None gcsfs : None matplotlib : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : None pyxlsb : None s3fs : None scipy : None sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None xlwt : None numba : None

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
mroeschkecommented, Aug 26, 2021

@lorenzophys yes that sounds like a good location

0reactions
lorenzophyscommented, Aug 25, 2021

@mroeschke I would like to write the tests for this behavior. Being my first issue with this codebase, tests related to this issue will go in pandas/tests/io/json/test_pandas.py? Or there’s a more appropriate place?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Unexpected ConvertTo-Json results? Answer: it has a default
Answer. ConvertTo-Json has a -Depth parameter: Specifies how many levels of contained objects are included in the JSON representation.
Read more >
Solve common issues with JSON in SQL Server
Question. I want to produce complex JSON with several arrays on the same level. FOR JSON PATH can create nested objects using paths,...
Read more >
Error 15360 when parsing json file with READ-JSON()
When reading a complex .json file using READ-JSON following error occurs: Error parsing JSON: unexpected token: bracket. (15360).
Read more >
An Exploration of JSON Interoperability Vulnerabilities
JSON Serialization Quirks; Float and Integer Representation; Permissive Parsing and Other Bugs. 1. Inconsistent Duplicate Key Precedence. Creed ...
Read more >
Unexpected Adventures in JSON Marshaling | CrowdStrike
In this blog post, we explore how adding seemingly benign new methods can actually be a breaking change, especially when working with JSON...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found