question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Package validation doesn't support non-object resource schema

See original GitHub issue

Overview

According to the documentation (https://specs.frictionlessdata.io/data-resource/#resource-schemas):

The value for the schema property on a resource MUST be an object representing the schema OR a string that identifies the location of the schema.

If a string it must be a url-or-path as defined above, that is a fully qualified http URL or a relative POSIX path. The file at the location specified by this url-or-path string MUST be a JSON document containing the schema.

I have a package where resources schemas are specified as URLs:

{
      "name": "deployments",
      "path": "deployments.csv",
      "profile": "tabular-data-resource",
      "format": "csv",
      "mediatype": "text/csv",
      "encoding": "utf-8",
      "schema": "https://raw.githubusercontent.com/tdwg/camtrap-dp/0.1.6/deployments-table-schema.json"
    },

Unfortunately, I cannot validate the package because frictionless-py requires schema to be an object:

The code:

import json
from pprint import pprint
from frictionless import validate_package

with open("example/datapackage.json") as json_file:
  descriptor_data = json.load(json_file)

  report = validate_package(descriptor_data)
    for err in report.errors:  # report.valid is False
      pprint(err)

The result:

{'code': 'package-error',
 'description': 'A validation cannot be processed.',
 'message': 'The data package has an error: '
            '"\'https://raw.githubusercontent.com/tdwg/camtrap-dp/0.1.6/deployments-table-schema.json\' '
            'is not of type \'object\'" at "resources/0/schema" in metadata '
            'and at '
            '"allOf/0/properties/resources/items/properties/schema/type" in '
            'profile',
 'name': 'Package Error',
 'note': '"\'https://raw.githubusercontent.com/tdwg/camtrap-dp/0.1.6/deployments-table-schema.json\' '
         'is not of type \'object\'" at "resources/0/schema" in metadata and '
         'at "allOf/0/properties/resources/items/properties/schema/type" in '
         'profile',
 'tags': []}

Thanks for your help!


Please preserve this line to notify @roll (lead of this repository)

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
peterdesmetcommented, Nov 24, 2021

Hi @roll, I would like to reopen this issue. I did not encounter this issue with older versions of frictionless (v4.18.2) and the bug is not related to the custom profile. I’ve created a minimal example:

  1. When I use "profile": "data-package" in my datapackage.json then the validation works fine, even when using a path (rather than a verbose object) in resource schema.
  2. When I use "profile": "https://specs.frictionlessdata.io/schemas/data-package.json" then the validation fails, saying that the schemas should be an object (and not a path string):

The data package has an error: “‘species-table-schema.json’ is not of type ‘object’” at “resources/0/schema” in metadata and at “allOf/0/properties/resources/items/properties/schema/type” in profile

Why the different behaviour? Note also that according to the specs, both path and object are allowed, but that the JSON schema has "schema": { "type": "object" } not "type": ["string", "object"]

0reactions
rollcommented, Nov 2, 2021

I think the problem is here - https://github.com/tdwg/camtrap-dp/blob/main/camtrap-dp-profile.json#L36. The custom profile doesn’t let the schema property be a string.

Closing for now, as it seems to be not related to the framework itself.

Please let me know if it didn’t help.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Json schema validation error - Stack Overflow
The schema provided is a slightly modified meta-schema, it validates JSON schema structure. The JSON data provided looks more like a DTO, ...
Read more >
simpl-schema/README.md at main - GitHub
A JavaScript schema validation package that supports direct validation of MongoDB ... Don't mix the object property definitions in with the main schema....
Read more >
Resource type schema - CloudFormation Command Line ...
To be considered valid, your resource type's schema must adhere to the Resource provider definition schema . This meta-schema provides a means of...
Read more >
SchemaFactory (Java SE 19 & JDK 19) - Oracle Help Center
SchemaFactory is a schema compiler. It reads external representations of schemas and prepares them for validation. The SchemaFactory class is not thread-safe.
Read more >
Validation with Hibernate Validator - Quarkus
Enable the Post group, meaning only constraints assigned to the Post (and Default ) groups will be validated for the book parameter of...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found