feature: Set Meltano system database_uri for different Meltano Environments in meltano.yml env block

Feature scope

Configuration (settings parsing, validation, etc.)

Description

I would like to be able to set a different Meltano system database_uri (this setting) under an env block in meltano.yml. Currently, the only way to change this setting across environments is to change the MELTANO_DATABASE_URI environment variable (&/or .env file variable).

This is not currently possible.

It would look something like this:

environments:
- name: dev
  config:
    database_uri: postgresql://test:[e-mail]:5432/postgres
    plugins:
      loaders: ...

Why is this a useful feature?

• Keeping the system db separate in different environments is helpful. I like to limit shared resources between environments bc sharing them makes accidentally overwriting from one environment to the other more likely)
• Allowing this setting to be included in meltano.yml makes it more visible and transparent to all users, and reduces the amount of manual differentiating that needs to be done in other places.

There is a discussion of this in Slack here: https://meltano.slack.com/archives/C01TCRBBJD7/p1664314402975189

[qbatten]

As a workaround, is it sufficient to specify this using an environment variable that you pre-configure before running Meltano CLI commands? The extra manual step of configuring the env variable is not ideal, but I’m curious if this makes a valid workaround for you for the time being. What do you think?

@aaronsteers yep, that’s what I’m doing right now and it works fine! This doesn’t feel critical to me, it just would be nice to have.

@tayloramurphy re: #6848, I havent run into any overwrites bc I was concerned about that happening and therefore decided to use different system DBs for dev and prod. I do think these two issues are sort of linked, in that they’re both about making environments work smoothly for users.

[aaronsteers]

Somewhat related, but it seems that the database_uri configuration option currently is also not expanding any $ variables from the .env or terminal env.

Good point.

Logged here:

• https://github.com/meltano/meltano/issues/6920