Support import of category type in DB Engine
See original GitHub issueFor following script
import pandas as pd
import pyarrow as pa
import sys
sys.setdlopenflags( 1|256 ) # RTLD_LAZY+RTLD_GLOBAL
from dbe import PyDbEngine as PDE
pdb = PDE("data", 8874)
data = {
"id1": ["id1", "id2", "id3", "id1", "id2", "id3", "id1", "id2", "id3", "id1"],
}
df = pd.DataFrame(data)
df["id1"] = df["id1"].astype("category")
a = pa.Table.from_pandas(df)
pdb.consumeArrowTable('testtable', a)
we currently have
terminate called after throwing an instance of 'std::runtime_error'
what(): dictionary<values=string, indices=int8, ordered=0> is not yet supported.
Aborted (core dumped)
this is h2o case, reproducible on https://github.com/intel-go/omniscidb/commits/consuming_arrow_table
Issue Analytics
- State:
- Created 3 years ago
- Comments:6 (6 by maintainers)
Top Results From Across the Web
Permissions (Database Engine) - SQL Server - Microsoft Learn
Base securable Granular permissions on base securable Securable that contains bas...
APPLICATION ROLE ALTER DATABASE
APPLICATION ROLE CONTROL DATABASE
APPLICATION ROLE VIEW DEFINITION DATABASE
Read more >Import data into an Amazon RDS DB instance - AWS
To import data from an existing database to an RDS DB instance: Export data from the source database. Upload the exported data.
Read more >What Is NoSQL? NoSQL Databases Explained - MongoDB
Brief History of NoSQL Databases; NoSQL Database Features; Types of NoSQL ... Some NoSQL databases like MongoDB do, in fact, support ACID transactions....
Read more >Common Issues with the SQL Server Import and Export Wizard
The SQL Server Import and Export Wizard is useful for copying data from one data source (e.g. a SQL Server database) to another....
Read more >How To Work with JSON in MySQL - DigitalOcean
Note: This will only work for database engines that support the JSON data type. Engines, such as older versions of MySQL will not...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
For now in case of import with arrow we can just omit conversion to category since FSI treats strings as encoded dictionary anyway
@anmyachev we are waiting a small fix from omnisci team. See https://github.com/modin-project/modin/issues/2747.