Define and implement a way to set right column types when auto-generating the database

I’m submitting a

• [x ] feature request.

Current Behaviour:

All database columns are set to db.String as Hydra do not require specifying the type of variable for a filed

Expected Behaviour:

Database columns should be created with dbString, db.Integer or db.Float according to types defined in the ApiDoc or Ontology via properties like "@type": " "xsd:decimal" as discussed in https://github.com/HTTP-APIs/creditrisk-poc/issues/6

I can work on this but I need support from @sameshl

[chrizandr]

Right now, hydrus assumes that every property is either a string literal or a URI pointing to a resource. In both cases, they are stored as strings in the database. So for example if we have a property for temperature that is supposed to have a number, hydrus doesn’t check if the value sent to it is a number or not. It will store a URI if we send a URI in that field.

When creating the tables in the database, we are not checking the @type field for a given supported property defined in the API documentation. We can implement this check when the database is being created or when the request/response is being processed. We say that we add support for the core RDF datatypes in hydrus. Whenever a supported property is a literal, it can be defined in the API documentation with one of these four types: “xsd:string”, “xsd:boolean”, “xsd:decimal”, “xsd:integer”.

So suppose a property is defined as:

{
"name": "Temperature",
"@type": "xsd:integer"
}

We cast the property as integer. Similarly for the other 3 datatypes.

In case the datatype is a custom type, it will be defined in the API documentation as :

{
"name": "Temperature",
"@type": "http://weather.com/schema/celsius"
}

In this case, the property is assumed to have instances of a resource of some existing ontology, and the field would then store only URIs pointing to instances of the resource, which may or may not be in the API. We do not need to derference the @type, because hydrus already assumes it to be an instance of some other resource.

Something similar is implemented here already: https://github.com/HTTP-APIs/hydrus/blob/34c909e26e58bf2aee279391c7560d8536ff9f91/hydrus/data/db_models.py#L94

Where the @type is checked to be a resource from the same API doc to add foreign key relations. We just need to add checks below for “xsd:string”, “xsd:boolean”, “xsd:decimal”, “xsd:integer”. All other cases would be treated as URIs in the database to be stored as strings.

You can ping me on slack if you need any help.

[Mec-iS]

Is there any use for us to store them as integer or floats when all hydrus is doing is serving the data?

There are major cons:

• it is against best practices to not respect types in an SQL database and it is a huge waste of memory. It is against the basics of database normalisation.
• JSON standard supports the types int and float, and therefore they must be represented with the right type. It would be basically misusing serialisation.
• payloads that deliver numbers as strings break conventions with all existing clients.