Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Option for importing CSV data using the SQLite .import mechanism
See original GitHub issueAs seen in https://til.simonwillison.net/sqlite/import-csv - .mode csv and then .import school.csv schools is hugely faster than importing via sqlite-utils insert and doing the work in Python - but it can only be implemented by shelling out to the sqlite3 CLI tool, it’s not functionality that is exposed to the Python sqlite3 module.
An option to use this would be useful - maybe something like this:
sqlite-utils insert blah.db blah blah.csv --fast
Issue Analytics
- State:
- Created 2 years ago
- Comments:22 (22 by maintainers)
Top Results From Across the Web
Import a CSV File Into an SQLite Table
Summary: in this tutorial, you will learn various ways to import CSV data into an SQLite table using sqlite3 and SQLite Studio tools....
Read more >SQLite - Import Data from a CSV File - Quackit Tutorials
To import data into SQLite, use the .import command. This command accepts a file name, and a table name. The file name is...
Read more >Simon Willison on Twitter: "TIL you can run SQL queries ...
The new sqlite-utils memory command can import CSV and JSON data directly into an in-memory SQLite database, combine and query it using SQL...
Read more >Feature request: Import csv into a table with generated columns
import creates INSERT statement(s) using the .csv (or .tsv) header to name incoming columns, without being picky about mapping them one-to-one ...
Read more >Joining CSV and JSON data with an in-memory SQLite ...
sqlite -utils already offers a mechanism for importing CSV and JSON data into a SQLite database file, in the form of the sqlite-utils...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Should consider how this could best handle creating columns that are integer and float as opposed to just text.
https://discord.com/channels/823971286308356157/823971286941302908/1019630014544748584 is a relevant discussion on Discord. Even if you create the schema in advance with the correct column types, this import mechanism can put empty strings in blank float/integer columns when ideally you would want to have nulls.
Related feature idea for
sqlite-utils transform:Not sure how best to handle this for
sqlite3 .importimports.Relevant TIL: https://til.simonwillison.net/sqlite/one-line-csv-operations