Add ability to provide custom `--state-id-suffix` to `meltano run`

One thing that was cut late in development when adding incremental job support to meltano run was the ability to add a custom prefix to the auto generated job (now state) names: https://gitlab.com/meltano/meltano/-/merge_requests/2544#note_855957749

I think we’ve agreed on this already but just want to reiterate given that we’ve just logged ADR: Refactor/rename “job_id” as “state-id” (#3290 - closed), it is probably best to try to avoid user config/override for the state_id_prefix or job_id_prefix. Just for the reason that if we allow them to configure the naming convention, and then we update what it is called, that will create a breaking change which I think it’s best to avoid.

Now that the job -> state transition work has basically wrapped up we could revisit adding this.

@tayloramurphy fwiw this is a pretty easy-peasy change on the dev side. If only configurable as a cli arg weights probably as low as a 2, but if we want to also support this as a per env meltano.yml option than most likely more realistically a weight of 4.

[tayloramurphy]

@aaronsteers looking at this proposal, yes I think it makes sense for the first 2 👍 Along with the spec that the suffix is delimited with : and we disallow ‘:’ from within the provided suffix and from within the component parts.

We’d need to spend a moment on how environment variable inheritance works to make sure it’s clear to user (similar to what Cody did in https://docs.meltano.com/guide/configuration#environment-variable-expansion ). Otherwise I like it

[aaronsteers]

@ReubenFrankel - Yes, exactly. And thanks for confirming regarding your use case related to #6164.

We could then take an approach of:

1. Add --state-id-suffix support in meltano run.
2. Add state_id_suffix: SUFFIX support in meltano.yml environment definitions (as sampled in @ReubenFrankel’s example code above), and support the setting of this via the meltano config CLI.
3. Long-term, we can continue to evaluate how these are used and what additional features we’d want (if at all) in a more ‘native’ meltano construct for something like “sub-environments” or “environment matrices”.

@tayloramurphy - From the product side, what do you think of this approach of these three steps, green-lighting the first two steps above? Basically, these would also be a solve for #6164, at least as written.

The only disadvantage I can see about this ordering, and not going directly into the meltano.yml approach, is that presumably this introduces some small additional risk of environments being misconfigured. I’d like to eventually land on a spec that gives a very high confidence that each config item in the matrix is properly set to match the others. That’s something though, I think we can continue to iterate upon.