overrideable autoescape

At work, we are using Jinja to render templates as YAML. We’d like to customize all {{ variable }} expansions to do something sensible to Python values like None (which is null in YAML) and empty string (which is encoded as '' in YAML). It seems like the Jinja autoescaping mechanism would be perfect for this. Unfortunately, I don’t see any way to configure the autoescaping functionality to not autoescape to HTML. Instead, I’ve had to monkeypatch Markup and escape in jinja2.runtime. It would be nice if there were an officially-sanctioned method to do this, for example by overriding something in the environment.

[ThiefMaster]

Using Jinja templates or any other string-based template language or string operations at all to generate JSON is just wrong. I can see why you would do this for YAML since it is meant to be human-friendly, too, but JSON should be generated from e.g. a Python dict and not from a Jinja template.

(that said, I am curious why you want to do that and what you are trying to do :p)

[glasserc]

The first part of my solution was the extension that runs all variable calls through a filter. This is the bare minimum that I needed to get that working.

import jinja2.ext
from jinja2.lexer import Token

class YAMLEverythingExtension(jinja2.ext.Extension):
    """
    Insert a `|yaml` filter at the end of every variable substitution.

    This will ensure that all injected values are converted to YAML.
    """
    def filter_stream(self, stream):
        # This is based on https://github.com/indico/indico/blob/master/indico/web/flask/templating.py.
        for token in stream:
            if token.type == 'variable_end':
                yield Token(token.lineno, 'pipe', '|')
                yield Token(token.lineno, 'name', 'yaml')
            yield token

This could be smarter, for example it could try to skip inserting the filter if it sees a safe or yaml filter explicitly in the variable block. But I never ended up needing/having time for that.

The second part is the filter itself. Just using yaml.dump wasn’t sophisticated enough, so I had to poke around the yaml internals a little bit.

import cStringIO
import yaml

def yaml_filter(val):
    """Serialize some value in isolation, not as part of any document.

    We can't just use yaml.dump because that outputs an entire document, including newlines, which isn't helpful for
    inserting into a YAML document."""
    if isinstance(val, jinja2.Undefined):
        val._fail_with_undefined_error()
    stream = cStringIO.StringIO()
    dumper = yaml.dumper.Dumper(stream)
    dumper.open()
    node = dumper.represent_data(val)
    dumper.serialize(node)
    # The serialized node tends to have a \n at the end.  The template might not
    # want a \n inserted here, e.g. if two variables are on the same line, so
    # strip.
    return stream.getvalue().strip()

This ties it all together, including making sure that the filter of the given name is available in the environment:

from jinja2 import loaders
from jinja2 import Environment, StrictUndefined

def get_environment():
    """Create a standard Jinja environment that has everything in it.
    """
    jinja_env = Environment(extensions=(YAMLEverythingExtension,),
                            # some other options that we use at work
                            loader=loaders.FileSystemLoader(['.', '/']),
                            undefined=StrictUndefined)
    jinja_env.filters["yaml"] = yaml_filter
    return jinja_env