Consul Tokens from Spring Vault do not get picked up by Spring Cloud Config Consul

I’m trying to get a consul token from vault, so that spring cloud consul config can get additional config from consul.

When I use this config:

# bootstrap.yml

## Config for vault
spring.cloud.vault:
  host: vault.host
  port: 443
  scheme: https # must be https for production
  config:
    lifecycle:
      enabled: true
    order: -10
  authentication: APPROLE # Same thing happens when using token
  app-role:
    role-id: ******
    secret-id: *******
  consul:
    enabled: true
    role: application
  fail-fast: false

## Consul config
spring.cloud.consul:
  enabled: true
  host: 127.0.0.1 
  port: 8500
  config:
    enabled: true
    format: FILES
    failFast: true
    profile-separator: '-'
    default-context: application

Here’s what I see on app startup:

2016-12-01 11:01:19.701 DEBUG 58221 --- [           main] o.s.c.e.PropertySourcesPropertyResolver  : Could not find key 'spring.cloud.consul.token' in any property source
2016-12-01 11:01:19.701 DEBUG 58221 --- [           main] o.s.c.e.PropertySourcesPropertyResolver  : Could not find key 'CONSUL_TOKEN' in any property source
2016-12-01 11:01:21.383 DEBUG 58221 --- [           main] org.apache.http.wire                     : http-outgoing-0 << "{"request_id":"3d26b618-1636-fa7b-2a1b-6039d3e4383f","lease_id":"consul/creds/application/570da8f4-7b23-6cf6-5956-8bcec0f9b735","renewable":true,"lease_duration":2592000,"data":{"token":"xxxxxxxxxx"},"wrap_info":null,"warnings":null,"auth":null}[\n]"
2016-12-01 11:01:21.854 DEBUG 58221 --- [           main] org.apache.http.wire                     :  >> "GET /v1/kv/config/application.properties?token= HTTP/1.1[\r][\n]"
2016-12-01 11:01:21.878 DEBUG 58221 --- [           main] org.apache.http.wire                     :  >> "GET /v1/kv/config/application.yaml?token= HTTP/1.1[\r][\n]"
2016-12-01 11:01:21.905 DEBUG 58221 --- [           main] org.apache.http.wire                     :  >> "GET /v1/kv/config/application.yml?token= HTTP/1.1[\r][\n]"
2016-12-01 11:01:21.920 DEBUG 58221 --- [           main] o.s.c.e.PropertySourcesPropertyResolver  : Found key 'spring.cloud.consul.token' in [bootstrapProperties] with type [String]

Note that while a vault token is obtained, Spring Cloud Consul Config does not seem to be picking it up.

Here’s the dependencies in play (With Spring boot 1.4.2.RELEASE)

[INFO] +- org.springframework.cloud:spring-cloud-consul-config:jar:1.1.2.RELEASE:compile
[INFO] +- org.springframework.cloud:spring-cloud-starter-consul-discovery:jar:1.1.2.RELEASE:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-starter-consul:jar:1.1.2.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-commons:jar:1.1.6.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-context:jar:1.1.6.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-consul-core:jar:1.1.2.RELEASE:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-consul-discovery:jar:1.1.2.RELEASE:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-netflix-core:jar:1.2.3.RELEASE:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-starter-ribbon:jar:1.2.3.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-starter:jar:1.1.6.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-starter-archaius:jar:1.2.3.RELEASE:compile
[INFO] +- org.springframework.cloud:spring-cloud-vault-starter-config:jar:1.0.0.M1:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-vault-config:jar:1.0.0.M1:compile
[INFO] +- org.springframework.cloud:spring-cloud-vault-config-consul:jar:1.0.0.M1:compile# cloud dependencies

Looking at the /env endpoint, the property is set, but apparently not in time, for consul config to pick it up.

[mp911de]

The reason why spring.cloud.consul.token is not picked up by ConsulConfigProperties is that both components (Spring Cloud Vault, Spring Cloud Config) provide PropertySource that are initialized first and then added to Spring’s Environment.

At the time of PropertySource initialization, we require ConsulConfigProperties to be initialized. ConsulConfigProperties happens with the Environment without Spring Cloud Vault’s PropertySource.

You can spawn an own bootstrap configuration to provide a Consul token obtained from Vault to the Environment in a very early stage. It’s a workaround based on Spring Vault API, see https://gist.github.com/mp911de/17f550ffecdc9e8f22061bfdf896bbb4

Make sure to place the bootstrap configuration class outside the packages that Spring Boot scans otherwise the configuration initializes twice. You also need a META-INF/spring.factories resource that points to the bootstrap config.

We will investigate further how we could simplify such use cases as another config component may contribute properties that could be required to use another config component (i.e. store the Consul token inside Spring Cloud Config to be used with Consul).

[mp911de]

Please note there are three years between 2016 and 2019. The arrangement remains unchanged as each component that wants to contribute a PropertySource for a @Bean method requires its own application context that is prepended to the actual application context. Creating more than the additional bootstrap context is impractical. Unless we can find a different approach, we cannot do anything here. Therefore, we’re closing this ticket with won’t fix.