question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

add_implicit_resolver does not match quoted strings

See original GitHub issue
import re
import uuid
import yaml

regex = re.compile(r'^UUID\((.+)\)$')
yaml.add_implicit_resolver('!uuid', regex)

def convert_uuid(loader, node):
    value = loader.construct_scalar(node)
    str_value = regex.match(value).group(1)
    return uuid.UUID(str_value)

yaml.add_constructor('!uuid', convert_uuid)

print(yaml.load('''
    config:
        abc: UUID(6a02171e-6482-11e9-ab43-f2189845f1cc)
        def: abc
'''))

# {'config': {'abc': UUID('6a02171e-6482-11e9-ab43-f2189845f1cc'), 'def': 'abc'}}

print(yaml.load('''
    config:
        abc: 'UUID(6a02171e-6482-11e9-ab43-f2189845f1cc)'
        def: abc
'''))

# {'config': {'abc': 'UUID(6a02171e-6482-11e9-ab43-f2189845f1cc)', 'def': 'abc'}}

print(yaml.load('''
    config:
        abc: !uuid 'UUID(6a02171e-6482-11e9-ab43-f2189845f1cc)'
        def: abc
'''))

# {'config': {'abc': UUID(6a02171e-6482-11e9-ab43-f2189845f1cc), 'def': 'abc'}}

Maybe this is something I don’t understand about the YAML spec, but if you explicitly tag the value it works. I initially thought it was #294, but I am using PyYAML 5.3.1 with Python 3.8.6.

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:1
  • Comments:9 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
ingydotnetcommented, Feb 5, 2022
# would prefer
someKey: @do_not_inherit someValue

There’s no reason you couldn’t do this now using:

someKey: !do_not_inherit someValue
# or:
someKey: !do_not_inherit [ !tag taggedValue ]

where the function you register to !do_not_inherit modifies the constructed value in some way desired by your framework.

Ruamel is a fork of PyYAML. I just took a quick look and all the resolver and constructor functions I mentioned in the previous comment are available in ruamel too.

1reaction
ingydotnetcommented, Feb 5, 2022

As @Thom1729 said

Tag resolution is the process of taking each node with a non-specific tag and assigning it a specific tag.

Tags themselves are annotation strings that can be used by the composition/construction process to configure that process. In reality (in PyYAML) tags are keys to look up functions. The actual functions that transform representation nodes into native structures.

For example: the !!bool tag is a key to look up a python function to create a python bool value.

This is the PyYAML code that implicitly tags plain scalar values matching certain patterns: https://github.com/yaml/pyyaml/blob/master/lib/yaml/resolver.py#L170-L175

to the tag tag:yaml.org,2002:python/bool which is configured here: https://github.com/yaml/pyyaml/blob/master/lib/yaml/constructor.py#L665-L667

To tell the constructor to transform the canonical form of the scalar using this function: https://github.com/yaml/pyyaml/blob/8cdff2c80573b8be8e8ad28929264a913a63aa33/lib/yaml/constructor.py#L233 to make the python boolean native value.

The process of turning a node into a canonical form is not well abstracted or exposed by pyyaml. In this boolean example, it simply calls .lower() on the value and uses that as a local dict key.


It is easy to apply multiple functions (tags) to a node’s construction, by wrapping calls in sequences. Using @Thom1729 's example, you can:

- !force-inherit [ !tag2 some string ]

it also makes the order of operations visually clear. And you can obviously do this right now in pyyaml; although we are working on ways to make it even cleaner.


On the subject of tag resolution and canonical form, this is usually seen in the context of scalars but you can also do this with collections.

For instance you shouldn’t have to tag !color for:

pink: !color { r: 255, g: 192, b: 203 }
blue: { r: 0, g: 0, b: 255 }

and the normalization should be equally powerful. This should be able to produce the same:

pink: r=255 g=192 b=203
blue: [0, 0, 255]

If a schema dictates that a given node in the graph is required to have the tag !color it should be able to express the normalization from various serialization forms as shown above.

A yaml schema is the entity that defines the tags available to a load operation and how the implicit resolution and canonicalization should work.

A schema should be able to be written as a yaml data file and exposed to a YAML API like pyyaml’s via something like:

native = yaml.load(yaml_input, schema_file='drawing-schema.yaml')

We’re not there yet, but we’re working on it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

regex for that excludes matches within quotes - Stack Overflow
I find that most matches that are not useful for my purpose are the matches that occur with strings within quotes, for example:...
Read more >
snakeyaml / Howto - Bitbucket
Step-by-step instructions for common tasks. How to substitute object in YAML document with a custom object.
Read more >
Regex for Quoted String with escapable quotes - Metal Toad
1. Match a single or double quote, as long as it's not preceded by \ · 2. Store that match in a way...
Read more >
Find quoted strings - The Modern JavaScript Tutorial
Please note, in particular, that an escaped quote \" does not end a string. So we should search from one quote to the...
Read more >
DEQUOTE Function - SAS Help Center
Removes matching quotation marks from a character string that begins ... Under certain circumstances, the I18N Level 1 functions might not ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found