_env cannot be set for multiple tasks

Scenario

I am using the mysql command line executable plus the sh operator to perform various MySQL operations in DigDag. mysql automatically checks for MYSQL_PWD in the environment for the password to use. I am trying to set that environment variable with a Digdag secret.

I’d like to set that once and have the environment set up for child tasks.

Complications

Root _env

Setting an _env at the .dig root gives an error.

_env:
  MYSQL_PWD: ${secret:mysql.password}

gives error:

Workflow 'td_to_mysql' includes unknown keys: [_env] (model validation)

Parent Task _env

+many_mysql_comments:
  _env:
    MYSQL_PWD: ${secret:mysql.password}
  +command_1:
    sh>: mysql --execture "${SQL}"
  +command_2:
    sh>: mysql --execture "${MORE_SQL}"

gives the error +td_to_mysql+many_mysql_comments contains invalid keys: '_env': "{"_env":{"MYSQL_PWD":"${secret:mysql.password}"}}" (config)

Question

What is the best way to set up the environment so multiple tasks can use it? Thanks!

[sonots]

I am also suffering with _env. I want to configure _env globally.

My scenario is like:

1. Set _export: docker: globally so that I can run all tasks in a docker container.
2. Want to get secrets in docker-entrypoint.sh from environment variables so that I can avoid embed secrets into docker images.
3. It requires me to map all secrets to environment variables using _env for all tasks.

If it is possible to configure _env globally with a giant task or with a global _export, my suffering should be relieved.

[hiroyuki-sato]

Hello, @friendofasquid

I’m thinking about another solution. I’ll let you know if I find it.

BTW, I’m maintaining mysql> operator. It uses secrets so You don’t need to set MYSQL_PWD environment variables.