Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow CSV resource schema to be string

See original GitHub issue

It seems that the base spec allows a resource’s schema to be an object or a string, but the jsontable and tabular data spec insists that it be an object.

Would it be reasonable to allow this to be a string? It would help keep the individual resource objects small when there are many separate resources (ie different years) with the same columns.

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:12 (10 by maintainers)

github_iconTop GitHub Comments

pwalshcommented, Jul 28, 2016

@danfowler yes, thanks.

@rgrp I think we should remove, at least for V1, the various options for setting a schema. It is too flexible. I’d prefer something simple and explicit: schema property on a resource can be an object (the schema) or a url (to a schema object as json).

rufuspollockcommented, Dec 1, 2016


To summarize the issue here as it got a bit confused: TDP was not clear that schema follows Data Package spec in allowing the value of schema property to be specified both inline and out of line. This has now been corrected. (Tabular Data Package was clear that schema value had to be a JSON Table Schema.)

Note: #295 etc is something different and can be considered quite separately.

Read more comments on GitHub >

github_iconTop Results From Across the Web

CSV Schema Language 1.2
This is to allow for future evolution of the CSV Schema language to ... This takes 1 or 2 inputs, the first is...
Read more >
CSV Files - Spark 3.3.1 Documentation
Property Name Default Scope sep, read/write encoding UTF‑8 read/write quote " read/write
Read more >
Specifying a schema | BigQuery - Google Cloud
Manually specifying schemas · Option 1: Use Add field and specify each field's name, type, and mode. · Option 2: Click Edit as...
Read more >
Configure schema inference and evolution in Auto Loader
The Apache Spark DataFrameReader uses different behavior for schema inference, selecting data types for columns in JSON and CSV sources based on ...
Read more >
from_csv function | Databricks on AWS
To keep corrupt records, an user can set a string type field named columnNameOfCorruptRecord in an user-defined schema. If a schema does not ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found