question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ValueError when reading dict with None

See original GitHub issue

I find that reading a dict

row =  {'a': [1], 'b':[None]}
ks.DataFrame(row)

ValueError: can not infer schema from empty or null dataset

but for pandas there is no error

row =  {'a': [1], 'b':[None]}
print(pd.DataFrame(row))

   a     b
0  1  None

I have tried setting dtype=np.int64 but this has not helped.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:2
  • Comments:19 (7 by maintainers)

github_iconTop GitHub Comments

4reactions
skndrgcommented, May 20, 2021

@ederfdias Here is a possible workaround. Specify converters like below:

import numpy as np

df_ks = koalas.read_excel(
   ...
   converters={i : (lambda x: str(x) if x else np.NaN) for i in range(30)} # Read first 30 columns as string     
3reactions
ederfdiascommented, Apr 15, 2021

JFYI… using read_csv function with a column without values I don’t receive any errors, but with an read_excel() the same error is raised.

Read more comments on GitHub >

github_iconTop Results From Across the Web

ValueError: dict contains fields not in fieldnames: None
I am writing a code that will filter values in a csv file and write relevant rows to one file, accepted but not...
Read more >
Source code for satpy.readers - Read the Docs
... files_matching or reader_names is not None: reader_dict[reader_name] = (reader, files_matching) if files_to_sort: raise ValueError("No matching readers ...
Read more >
TypeError: 'NoneType' object is not subscriptable
NoneType is the type of the None object which represents a lack of value, for example, a function that does not explicitly return...
Read more >
ValueError if using all scalar values, you must pass an index
This error occurs as Pandas is expecting the data values to be list values or dict values. According to the code mentioned above,...
Read more >
Data Structures — Werkzeug Documentation (1.0.x)
If not further specified None is returned. type – A callable that is used to cast the value in the MultiDict . If...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found