Programmatically configure `Flags` via SPI

The default value of some Flags could be changed by only JVM -Dcom.linecorp.armeria.XXX option such as -Dcom.linecorp.armeria.verboseSocketExceptions=true. New options are constantly added to Flags according to various requirements and environments. Consequently, users find it difficult to override many options only via JVM options. They may not want to frequently fix the options in their CI/CD tools.

If we provide an SPI interface and let users programmatically override the default value by implementing a configuration interface. Users can take advantage of the benefits of reviewing the properties in the PR and managing it in the source set.

The following priority could be applied to the new configuration mechanism.

1. Give JVM options the highest priority.
2. If a JVM option is not specified, checks the value in the configuration provided via SPI
3. Neither is set, the sensible default defined in Flags will be used.

[ikhoon]

I want to use @ClearSystemProperty and @SetSystemProperty to set and clear system properties

System properties are loaded once when Flags is initialized. So @ClearSystemProperty and @SetSystemProperty do not effectively change the default values of Flags. We can use a custom class loader to re-evaluate the system properties. https://github.com/line/armeria/blob/4edffc8c11bd74665d652a1fd108d6a0307b6733/core/src/test/java/com/linecorp/armeria/common/FlagsTest.java#L92

There are 2 options in Flags that marked as deprecated( annotatedServiceExceptionVerbosity() and useEpoll() ). Should I still add them as part of the ArmeriaOptions SPI interface, or ignore them?

It is OK to exclude the deprecated flags from ArmeriaOptions.

[ikhoon]

I was thinking of the first approach in order to easily override the default options. For example:

public interface ArmeriaOptions {
    default int numCommonBlockingTaskThreads() {
        return 0; //return default value
    }

    default long defaultMaxRequestLength() {
        return 2; //return default value
    }
    ...
}

//Users implementation
class CustomArmeriaOptions extends ArmeriaOptions {
    @Override
    public long defaultMaxRequestLength() {
        return 3;
    }
}

class Flag {
    // Dynamically load AmeriaOptions
    static final ArmeriaOptions ARMERIA_OPTIONS = ServiceLoader.load(...);
   
    private static final long DEFAULT_MAX_REQUEST_LENGTH =
            getLong("defaultMaxRequestLength",
                    armeriaOptions.defaultMaxRequestLength(),
                    value -> value >= 0);

    public static long defaultMaxRequestLength() {
        return DEFAULT_MAX_REQUEST_LENGTH;
    }
}

The API consistency between Flag and ArmeriaOptions can be ensured with a test using reflection.