Proper Variable Expansion

Currently, dotbot uses os.path.expandvars. However, in some cases, it may be desireable to follow proper shell parameter substitution. (see: the related manual) Here is an example usage: I want my zshrc to work. Some systems may define $ZDOTDIR to be $XDG_DATA_HOME/zsh, some may leave it undefined. The typical solution would be to use ${ZDOTDIR:-$HOME}/.zshrc as the install location. However, without parameter substitution, there is no way to achieve this.

[simonvanderveldt]

FYI there would definitely be more projects that could use a way to use a POSIX compatible variable substitution, most of the time the main reason is being able to define a default value. Example of a project that’s looking for a solution to this docker/compose#1377

[anishathalye]

I’d prefer to write an implementation from scratch in pure Python.

For reference, here’s the implementation of posixpath.expandvars from Python 3.4:

# Expand paths containing shell variable substitutions.
# This expands the forms $variable and ${variable} only.
# Non-existent variables are left unchanged.

_varprog = None
_varprogb = None

def expandvars(path):
    """Expand shell variables of form $var and ${var}.  Unknown variables
    are left unchanged."""
    global _varprog, _varprogb
    if isinstance(path, bytes):
        if b'$' not in path:
            return path
        if not _varprogb:
            import re
            _varprogb = re.compile(br'\$(\w+|\{[^}]*\})', re.ASCII)
        search = _varprogb.search
        start = b'{'
        end = b'}'
        environ = getattr(os, 'environb', None)
    else:
        if '$' not in path:
            return path
        if not _varprog:
            import re
            _varprog = re.compile(r'\$(\w+|\{[^}]*\})', re.ASCII)
        search = _varprog.search
        start = '{'
        end = '}'
        environ = os.environ
    i = 0
    while True:
        m = search(path, i)
        if not m:
            break
        i, j = m.span(0)
        name = m.group(1)
        if name.startswith(start) and name.endswith(end):
            name = name[1:-1]
        try:
            if environ is None:
                value = os.fsencode(os.environ[os.fsdecode(name)])
            else:
                value = environ[name]
        except KeyError:
            i = j
        else:
            tail = path[j:]
            path = path[:i] + value
            i = len(path)
            path += tail
    return path

So it’s not way too complicated, but it’s not super simple either.