Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

KeyError: 'Nullable(String)'

See original GitHub issue

I get the following error with one column, even thoI do have other empty columns which do notcause any errors (such as aircraftCategory is a string and is NaN)

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-19-d531a02e1a3b> in <module>()
      2 
      3 connection = {'host': 'http://localhost:8123'}
----> 4 df = ph.read_clickhouse('SELECT hex, flight, recorded_date, recorded_datetime,recorded_longitude, recorded_latitude, recorded_squawk, recorded_altitude, recorded_verticalRate,recorded_track,recorded_speed,aircraftCategory FROM flightLog_MT LIMIT 1000000', connection=connection)

~/.pyenv/versions/3.6.2/envs/general_362/lib/python3.6/site-packages/pandahouse/core.py in read_clickhouse(query, tables, index, connection, **kwargs)
     56     lines = execute(query, external=external, stream=True,
     57                     connection=connection)
---> 58     return to_dataframe(lines, **kwargs)
     59 
     60 

~/.pyenv/versions/3.6.2/envs/general_362/lib/python3.6/site-packages/pandahouse/convert.py in to_dataframe(lines, **kwargs)
     60     dtypes, parse_dates, converters = {}, [], {}
     61     for name, chtype in zip(names, types):
---> 62         dtype = CH2PD[chtype]
     63         if dtype == 'object':
     64             converters[name] = decode_escapes

KeyError: 'Nullable(String)'

Issue Analytics

State:
Created 6 years ago
Reactions:3
Comments:14 (5 by maintainers)

Top GitHub Comments

1reaction

kszucscommented, Mar 6, 2019

@inkrement I’d suggest to use clickhouse-driver or https://github.com/ibis-project/ibis @haakonvt Sadly I don’t have time to implement it, however Ibis has support for it. I highly recommend to try ibis until We have a native Apache Arrow database interface for clickhouse.

0reactions

viralettacommented, Nov 18, 2020

I had rather similar problem

File "/opt/airflow/venv/lib64/python3.6/site-packages/pandahouse/core.py", line 58, in read_clickhouse
return to_dataframe(lines, **kwargs)
File "/opt/airflow/venv/lib64/python3.6/site-packages/pandahouse/convert.py", line 67, in to_dataframe
dtype = CH2PD[chtype]

It solved, when I deleted all comments from query, comments like “-- Убрать это” don’t work correctly in pandahouse