Table Schema v1: Add preface

There are some basic concepts which are used throughout the spec but are never explained. I think we should add a preface to the spec explaining its purpose, some basic assumptions and concepts that a reader should understand before diving deep into the details.

It should go along the lines of these bullet points:

Tabular Data has a logical representation - rows, columns, fields etc… Each field has a name and a data type (string, number etc.) When loading the data from a physical representation (e.g. file on disk), the data may have some data type information (e.g. JSON) or none (e.g. CSV), where all data is represented in string form.

The spec is about the logical representation of tabular data, as well as about how to convert a physical representation into the logical one:

• format (and accompanying properties) contain rules on converting from physical to logical representations
• constraints contain rules for validating the physical representation of the data.

By reusing these concepts (e.g. “missingValues works on the physical representation”) it should be much clearer than current wording.

[pwalsh]

@rufuspollock @akariv @roll

A text for the preface. I suggest adding this to the existing “concepts” section in the introduction. The current content of concepts could be titled “Tabular data” and this section title “Physical and logical representation of data”

---

Physical and logical representation data

In order to talk about the representation and processing of tabular data from text-based sources, it is useful to introduce the concepts of the physical and the logical representation of data.

The physical representation of data refers to the representation of data as text on disk, for example, in a CSV or JSON file. This representation may have some type information (JSON, where the primitive types that JSON supports can be used) or not (CSV, where all data is represented in string form).

The logical representation of data refers to the “ideal” representation of the data in terms of primitive types, data structures, and relations, all as defined by the specification. We could say that the specification is about the logical representation of data, as well as about ways in which to handle to conversion of a physical representation to a logical one.

In this document, we’ll explicitly refer to either the physical or logical representation in places where it prevents ambiguity for those engaging with the specification, especially implementors. For example, constraints should be tested on the logical representation of data, whereas a property like missingValues applies to the physical representation of the data.

[rufuspollock]

@pwalsh looks great and either a direct addition to the concepts section (maybe a subheading) or a separate subsection of the intro.

Assigning you for the glory of the PR 😉