Inconsistent connection update behavior, potentially causing client_secret deletion

Description

deploying config a0deploy deploy -c tenant/config.json --input_file tenant/tenant.yaml

results in error:

error: Problem running command deploy during stage processChanges when processing type connections error: Problem updating connections {“name”:“###”,“id”:“###”} Bad Request: Before creating Windows Azure Active Directory connections you must configure the client_id and client_secret settings in the dashboard. For more information go to https://docs.auth0.com/waad-clientid

also:

error: Problem running command deploy during stage processChanges when processing type connections error: Problem updating connections {“name”:“####”,“id”:“####”} Bad Request: The signing certificate is not valid. Please provide a .pem or .cer certificate base64 encoded public key

Reproduction

1. Export an existing environment, including one or more AzureAD connections

We are stripping out some sensitive information on export

config.json:

{
...
  "AUTH0_ALLOW_DELETE": false,
  "AUTH0_EXPORT_IDENTIFIERS": false,
  "AUTH0_EXCLUDED_RULES": [ "rule-1-name" ],
  "AUTH0_EXCLUDED_CONNECTIONS" : [
    "####",
    "smoke-test"
  ],
  "EXCLUDED_PROPS": {
    "connections": [
        "options.client_secret",
        "options.admin_access_token",
        "options.admin_access_token_expiresin",
        "options.cert",
        "options.signingCert"
    ]
  },
    "INCLUDED_PROPS": {
      "rulesConfigs": [ "key" ]
  }
}

tenant.yaml:

connections:
  - name: ####
    strategy: waad
    enabled_clients:
      - ####
    is_domain_connection: false
    options:
      domain: ####.com
      granted: true
      tenantId: ####
      client_id: ####
      use_wsfed: false
      app_domain: ####.auth0.com
      ext_groups: true
      ext_profile: true
      thumbprints:
        - ####
        - ####
        - ####
        - ####
        - ####
        - ####
      identity_api: azure-active-directory-v1.0
      basic_profile: false
      tenant_domain: ####.com
      waad_protocol: openid-connect
      domain_aliases:
        - ####.com
      api_enable_users: false
      ext_nested_groups: true
      useCommonEndpoint: false
      should_trust_email_verified_connection: ####
    show_as_button: false

Deploy the config: a0deploy deploy -c tenant/config.json --input_file tenant/tenant.yaml

the deployment halts with the given error.

The strange thing is that this worked perfectly in 2 other environments, with almost identical configurations (azuread and samlp connections) just a few days ago. Re-testing those environments they now throw the same errors. It looks like these options are now required instead of optional.

Workaround:

for now, you can add the failing connections to the AUTH0_EXCLUDED_CONNECTIONS list to skip them entirely

Environment

• Version of this library used: latest = 7.3.7

[willvedd]

This was also a big thing for me, I had assumed they were bi-directional and the documentation makes no mention that it will only be adhered to on import.

My understanding is that this only affects properties nested under the options property for connections. This is because most other endpoints adhere to normal PATCH conventions where excluded properties are ignored. However, as we can see in the endpoint docs, the options property does not adhere to that convention. So for most other circumstances it turns out to work bi-directionally.

[willvedd]

@lundybernard @kanocarra thanks for being patient. We’ve addressed this in our 7.5.0 release, I encourage you folks to update and test out.

The approach taken can be looked at more closely in #430 but in brief, we’ve just added excluded properties back onto the PATCH payload, which makes the bi-directional exclusion more explicit. We did consider the __IGNORE__ syntax, but it would’ve added much more surface area to the problem when this only affects connections. So for the sake of simplicity and backwards compatibility, we decided against it for now.

I appreciate the patience and feedback on this issue; don’t ever want to have the usage of this tool result in destructive changes. Going to close this out but please raise another issue if you’re experiencing any issues with the new release.