Add custom functions to the Jinja context

Problem

Programming in Jinja is not much fun, so if we need to do more complicated functions, the code can be quite complicated (or impossible).

Take this example (which doesn’t work and just frustrates the user):

{% set versionformat = lambda ver: '.'.join(ver) %}
version = {{ versionformat(31) }}

Proposal

It would be great if I could have files (*.conda-devenv.py) with:

def versionformat():
    """Convert string `ver` to a semantic version formatted string if not already"""
    # Case 1: if ver is '372' (string without dots), return '3.7.2'
    # Case 2: if ver is '3.7.2' (string with dots), return ver (in this case, '3.7.2')
    # Case 3: if ver is None, return None
    return '.'.join(ver) if ver and '.' not in ver else ver

Then in my environment.devenv.yml:

version = {{ versionformat(31) }}

Implementation details

• We should export all symbols found in __all__.
• I guess we should be using importlib.import_module to load the file(s).

Questions

1. Do we want 1 single file (conda-devenv.py) or all files in the current directory (*.conda-devenv.py) ?
2. Should all files be loaded automatically (implicit) or should them be loaded explicitly through an exposed function (load_functions("custom-functions-conda-devenv.py")) ?
3. What about dependencies between these files? Do we care about that? Can one file import another?

Why???

This feature request is based on the discussion held in https://github.com/ESSS/conda-devenv/pull/83. If conda-devenv had this capability, then the user would never feel the need to open that PR, he would just write his custom functions.

[nicoddemus]

What if we at least supported inline lambda functions in comments

I’m afraid inventing our own mini-language, or parsing the file ourselves and generating Python code, is not a good direction. We now have to deal with scopes or supporting single-line functions, which limits its usability. We would be supporting “jinja + our own extension language”. We would need to parse the file first, execute Python code to generate the lambda functions, and inject it later on the Jinja2 namespace. I’m afraid I’m a 👎 on this idea.

If we want to extend what’s available in the Jinja2 namespace, I think using plain Python modules is the way to go, with an environment.devenv.py file in the same directory.

But before we go down that route, I would like to see good use cases first, because we have been using conda-devenv for a good 2 or 3 years now, and have not come across the need for complicated functions. The recent addition of the prepreocessor selectors was something we could have been using for sure, but it is a simple preprocessing line with very reduced scope.

[tadeu]

I have mixed feeling about this. It’s understandable the motivation to allow users to add their own functions, but, on the other hand, environment.devenv.yml files are supposed to be as descriptive as possible, i.e., just plain data, with jinja2 acting as a “facilitator” to select some things given environment variables, and to remove some duplication of data.

Let’s continue the discussion, but keeping in mind that opening the door to custom user code that can execute anything and import anything would open the possibility of creating extra-complicated workflows.