Allow CSV resource schema to be string
See original GitHub issueIt seems that the base spec allows a resource’s schema
to be an object or a string, but the jsontable and tabular data spec insists that it be an object.
Would it be reasonable to allow this to be a string? It would help keep the individual resource objects small when there are many separate resources (ie different years) with the same columns.
Issue Analytics
- State:
- Created 7 years ago
- Comments:12 (10 by maintainers)
Top Results From Across the Web
CSV Schema Language 1.2
This is to allow for future evolution of the CSV Schema language to ... This takes 1 or 2 inputs, the first is...
Read more >CSV Files - Spark 3.3.1 Documentation
Property Name Default Scope
sep, read/write
encoding UTF‑8 read/write
quote " read/write
Read more >Specifying a schema | BigQuery - Google Cloud
Manually specifying schemas · Option 1: Use Add field and specify each field's name, type, and mode. · Option 2: Click Edit as...
Read more >Configure schema inference and evolution in Auto Loader
The Apache Spark DataFrameReader uses different behavior for schema inference, selecting data types for columns in JSON and CSV sources based on ...
Read more >from_csv function | Databricks on AWS
To keep corrupt records, an user can set a string type field named columnNameOfCorruptRecord in an user-defined schema. If a schema does not ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@danfowler yes, thanks.
@rgrp I think we should remove, at least for V1, the various options for setting a
schema
. It is too flexible. I’d prefer something simple and explicit:schema
property on a resource can be an object (the schema) or a url (to a schema object as json).FIXED.
To summarize the issue here as it got a bit confused: TDP was not clear that
schema
follows Data Package spec in allowing the value of schema property to be specified both inline and out of line. This has now been corrected. (Tabular Data Package was clear thatschema
value had to be a JSON Table Schema.)Note: #295 etc is something different and can be considered quite separately.