[Feature Request] Improving Microsoft.IdentityModel Observability

Improving Microsoft.IdentityModel Observability

There have been some internal asks for Microsoft.IdentityModel and dependent libraries to offer a better observability surface. Currently logging is possible but good hooks for metrics and thus alerting aren’t as readily available.

Observability areas to consider

• Logging signals to monitor
  • Additional logs that can clarify behavior for consumers
  • Metrics/Counters which can hopefully represent some dimensional slices
• System Resources to Monitor
  • While the library won’t be able to monitor its own system resource usage, it may be able to emit information that will allow consumers to better correlate operations with resource consumption.
• Anything which could assist in making Safe Deployment Practices more straightforward for services which consume the library
• Alerting
  • Addition of metrics/counters will likely make this easier however anything else that can make alerting easier should be considered

Areas of focus for observability

Some metrics/counters around time taken in validation and its dependencies (crypto calls etc.) as well as information about internal caching crypto cache/LRU info.

Means of communicating the logs/metrics

Microsoft.IdentityModel.Tokens.CallContext would appear the most promising place since it already contains the log collection.

Another possibility would be to extend scenario specific return values, e.g. TokenValidationResult so they could be decorated with metrics/logs specific to the scenario. I’m currently partial to using CallContext.

[kevinchalet]

@TimHannMSFT thank you for reaching out!

One of the biggest complaints I have with IdentityModel regarding logging is the fact that the error messages became useless OOTB since the introduction of the PII protection (that is enabled by default). Messages like IDX10205: Issuer validation failed. Issuer: 'System.String'. Did not match: validationParameters.ValidIssuer: 'System.String' or validationParameters.ValidIssuers: 'System.String' are cryptic for users trying to debug them. It’s a huge source of bug reports for libraries built on top of IdentityModel, like OpenIddict.

Potential ways to improve that:

• https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/pull/1741 helped a bit but most exception messages are still crude and don’t clearly mention that “System.String” values are actually redacted data that can be logged by using the static ShowPII property. I’d recommend making the exception messages more explicit about this option.
• Making PII logging configurable per token handler instance would be a great way to allow libraries leveraging IM to enable or disable it in a non-static way. In OpenIddict, this is typically the kind of thing that could be leveraged to include PII in log messages when IHostEnvironment.EnvironmentName indicates a development environment.

[Tratcher]

@Tratcher / @kevinchalet , was #1725 helpful to that end. Agreed we still have improvements to be made around this but hopefully that was a step forward on this journey, there’s a message which points folks towards https://aka.ms/IdentityModel/PII as well as no longer stripping the arguments that really aren’t PII.

It’s a start. I think some visual changes like those suggested above would help make this more understandable.

https://aka.ms/IdentityModel/PII should also give an example of an error message with PII on vs off so people understand what to expect.