Proposal: Resource Exclusion

• Start Date: March 14, 2022
• Github Issue: Proposal: Resource Exclusion · Issue #451

Summary

We propose a mechanism for excluding entire resources from the management purview of Deploy CLI.

Resource exclusion is an often requested feature from the community and has many legitimate use-cases. Currently, there is some resource-specific exclusion functionality, on a per-ID basis, but nothing that excludes entire resource types.

Why Exclude?

Excluding entire resources are useful for developers who work on tenants with many of a certain type. In the case of hundreds or thousands of a type of resource, it may be operationally inconvenient to manage all of those configurations.

Additionally, exclusion is useful for organizations who provision certain resource types manually or through different API means, and they do not wish to automate that resource type under the purview of deploy CLI.

It is understood that exclusions operate bi-directionally, meaning they work both on import and export. Currently, there is no clear use case for excluding resources for only one direction.

Existing Methods of Exclusion #

The following are existing configuration values for excluding resources on a per-ID basis:

• AUTH0_EXCLUDED_RULES
• AUTH0_EXCLUDED_CLIENTS
• AUTH0_EXCLUDED_DATABASES
• AUTH0_EXCLUDED_CONNECTIONS
• AUTH0_EXCLUDED_RESOURCE_SERVERS

There are key issues these configurations though:

1. They only apply on a per-ID basis, meaning that there is no way to automatically exclude all of that certain resource. This is a minor inconvenience for a handful of a certain resource, however, in the case of hundreds or thousands of a certain resource, it becomes untenable. Further, as more of a certain resource gets added, they’ll necessarily need to be added to the Deploy CLI configuration, requiring more touch points.

2. This only covers a subset of all resources. The Auth0 Management API is still expanding in scope, and exclusions for each resource Granted, it is possible to accommodate all n number of resource types with n number of exclusion properties, but can become unmanageable

3. Many entities may rely upon another specific resource in a complex web of dependencies. Each specific dependency is difficult to manage within the tool. For the simplicity and reliability of operation, it would be better to remove per-ID exclusions in favor of broader exclusions that can be more predictably managed.

Deprecation

The intention is to eventually deprecate the smaller, bespoke, exclusion configurations.

The reasoning is, that the Deploy CLI is positioned as a bulk configuration management tool. While configurable to omit certain resources, it is best suited for migrating large sets of tenant configuration. For more bespoke and granular control over the purview of configuration management, the Auth0 Terraform Provider may be a more appropriate tool for your workflow.

Further, omitting specific IDs from the management of this tool can have unintended and seemingly unpredictable consequences. Especially in the case of an entity depending on the existence of another.

Deprecation will not be immediate and will begin initially by displaying warning notifications in the console. Developers will be encouraged to adopt the broader method of exclusion proposed in this document.

Proposed Solution

The proposed solution is to introduce an AUTH0_EXCLUDED property in the user-defined JSON configuration file. This property will be an array of strings which contain the names of the resources that are intended for exclusion, providing the flexibility to target any and all resources.

An example of a user-defined JSON configuration file:

{
    "AUTH0_DOMAIN": "<DOMAIN>",
    "AUTH0_CLIENT_ID": "<CLIENT_ID>",
    "AUTH0_CLIENT_SECRET": "<CLIENT_SECRET",
    "AUTH0_ALLOW_DELETE": false,
    "AUTH0_EXCLUDED": [
        "connections",
        "rules",
        "actions",
    ]
}

Future Extensibility

Worth noting is that the proposed AUTH0_EXCLUDED property does yield a clear path for excluding specific IDs within a certain resource type in the future. This decision is intentional and meant to more clearly position this application as a bulk configuration management tool.

In the future however, it may be possible for this tool to include a AUTH0_INCLUDED property in combination with a wildcard operator available for AUTH0_EXCLUDED. Example:

{
    "AUTH0_EXCLUDED": "*",
    "AUTH0_INCLUDED": [
        "connections",
        "rules",
        "actions",
    ]
}

This would open the possibility to flip application to an opt-in paradigm as opposed to the current opt-out behavior.

Relevant Github Issues

• Provide a way to exclude all of an object during a0deploy import · Issue #375

• Add config to exclude Actions - AUTH0_EXCLUDED_ACTIONS · Issue #397

• Ability to exclude objects when import/export using deploy-cli · Issue #394

• Allow skipping export of Organizations · Issue #427

[mendhak]

I’d like to echo @MGough comments above (regarding the surprise!)

The reasoning is, that the Deploy CLI is positioned as a bulk configuration management tool.

This isn’t how it was positioned when we started using it, we were specifically told by Auth0 TAMs that this was the best way to manage resources both individually and in bulk, so I don’t see where the ‘bulk only’ reasoning has come from.

Further, switching to another method (the Terraform one), just for the sake of a few exclusions, is a non trivial task. This tool comes with its own workflows, environmental substitutions, configuration, multi-tenant deployments, The Terraform provider has its own workflow and also introduces the overhead of state management.

IMO this new positioning has not been communicated (or communicated well). Could the existing exclusion types be kept for the sake of backward compatibility?

[MGough]

Sorry for the delay in getting to review this proposal. I appreciate that Auth0 have started to assign time for the development of these libraries, and make it a viable product.

The reasoning is, that the Deploy CLI is positioned as a bulk configuration management tool. While configurable to omit certain resources, it is best suited for migrating large sets of tenant configuration. For more bespoke and granular control over the purview of configuration management, the Auth0 Terraform Provider may be a more appropriate tool for your workflow.

This is news to me! Previously the Deploy CLI seemed to be positioned as a general CI/CD tool. When I started using it nearly a year and a half ago there wasn’t much of an alternative that I could find. The terraform provider may have existed but when I have seen it in the past it wasn’t tracking with feature development, and it wasn’t managed by the Auth0 team. So you may find others who are surprised by this.

As a slight aside… One of the strengths of this tool compared to the terraform alternative was being able to allow a development tenant to be managed via the dashboard, then exporting the config, storing the alterations in source control, and knowing that it would work when applied to another environment/tenant. With terraform it’s trickier as you need to work out which resources that you need before you can import them. You can see how I managed it for our team using yarn scripts in my demo repo: https://github.com/MGough/auth0-trunk-based-deployments

As far as the proposal goes, it looks reasonable and hopefullly addresses the issues others face. It most likely signals the start of the end of my time using this tool. It makes sense not to have two different tools for the same purpose, I’ll have to see about getting us migrated across to terraform for Auth0.