question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support import of category type in DB Engine

See original GitHub issue

For following script

import pandas as pd
import pyarrow as pa
import sys
sys.setdlopenflags( 1|256 )    # RTLD_LAZY+RTLD_GLOBAL
from dbe import PyDbEngine as PDE

pdb = PDE("data", 8874)
data = {
        "id1": ["id1", "id2", "id3", "id1", "id2", "id3", "id1", "id2", "id3", "id1"],
    }
df = pd.DataFrame(data)
df["id1"] = df["id1"].astype("category")
a = pa.Table.from_pandas(df)
pdb.consumeArrowTable('testtable', a)

we currently have

terminate called after throwing an instance of 'std::runtime_error'
  what():  dictionary<values=string, indices=int8, ordered=0> is not yet supported.
Aborted (core dumped)

this is h2o case, reproducible on https://github.com/intel-go/omniscidb/commits/consuming_arrow_table

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
Garra1980commented, Jul 13, 2020

For now in case of import with arrow we can just omit conversion to category since FSI treats strings as encoded dictionary anyway

0reactions
fexolmcommented, May 6, 2021

@anmyachev we are waiting a small fix from omnisci team. See https://github.com/modin-project/modin/issues/2747.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Permissions (Database Engine) - SQL Server - Microsoft Learn
Base securable Granular permissions on base securable Securable that contains bas... APPLICATION ROLE ALTER DATABASE APPLICATION ROLE CONTROL DATABASE APPLICATION ROLE VIEW DEFINITION DATABASE
Read more >
Import data into an Amazon RDS DB instance - AWS
To import data from an existing database to an RDS DB instance: Export data from the source database. Upload the exported data.
Read more >
What Is NoSQL? NoSQL Databases Explained - MongoDB
Brief History of NoSQL Databases; NoSQL Database Features; Types of NoSQL ... Some NoSQL databases like MongoDB do, in fact, support ACID transactions....
Read more >
Common Issues with the SQL Server Import and Export Wizard
The SQL Server Import and Export Wizard is useful for copying data from one data source (e.g. a SQL Server database) to another....
Read more >
How To Work with JSON in MySQL - DigitalOcean
Note: This will only work for database engines that support the JSON data type. Engines, such as older versions of MySQL will not...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found