Support dependency injection into sinks, enrichers, filters, etc.

Assuming, we’d like to initialize Serilog in the first few lines of Program.Main(), and independently of ASP.NET Core/other hosting code, how can we support sinks and other pipeline elements that depend on the framework and components that are initialized later?

For example, in #84 there’s a discussion regarding the use of Application Insights configuration from dependency injection in a Serilog sink.

LateInitSink in this gist demonstrates a possible approach:

        var signalRSink = new LateInitSink();

        Log.Logger = new LoggerConfiguration()
            .WriteTo.Sink(signalRSink)
            .CreateLogger();

        Log.Information("Sadly, nobody will get this");

        // ... resolve that hub ...
        signalRSink.Init(wt => wt.Console());

        Log.Information("Ah so nice to be loggin' again");

@khellang suggested using a name like DependencyInjectedSink instead of LateInitSink, so make the API more searchable/discoverable.

We could, finalize/ship an implementation of this approach.

The need to also support enrichers, filters, and so-on, could be met by including a range of similar types for those cases.

As an alternative, or extension of this idea, we could create a wrapper API that mimics LoggerConfiguration and sets up the full range of extension points for DI (rough sketch):

// Initialize what we can, early on. The `deferred` variable
// will hold a `DeferredLoggerConfiguration` that's hooked
// into late-init sinks, enrichers, etc. by a new `CreateLogger()`
// extension-based overload.
Log.Logger = new LoggerConfiguration()
    .WriteTo.Console()
    .CreateLogger(out var deferred);

// Later, post-DI-setup, or elsewhere in the app:
deferred
    .AddSink(writeTo => writeTo.File(...))
    .AddEnricher(enrich => enrich.WithProperty(...))
    .UpdateLogger();

This API should be possible using some extension points already in 2.9.0; it would be nicer to support deferred.WriteTo.File() etc. but we’d need new APIs to enable it.

Thoughts?

See also #130.

[davidfowl]

                    var provider = builder.Services.BuildServiceProvider();

That line will create a different DI container than the one the application uses. You’ll end up with 2 sets of singletons and nobody is disposing the container here either so your disposable objects don’t get disposed.

[ralphhendriks]

Over the last days I have been struggling to figure out the current most correct way of configuring the Application Insights sink in an ASP.NET Core 3.1 application, preferrably without deprecation warnings. Reading serilog/serilog-sinks-applicationinsights#121 and related brought me to this issue.

I am all for a DI-friendly API design. However, I would be very reluctant to develop serilog’s API design to contain any explicit notion of DI, and MS.DI in particular. Several third party DI containers are non-conforming to the MS.DI interfaces, such as https://github.com/simpleinjector/SimpleInjector, whose main author @dotnetjunkie is actively involved in many discussion around this topic. With that in mind, I am not a fan of the proposed .AddAspNetCoreDIEnricher() API suggestions.

The main problem is quite well summarized by @cmeeren:

• The Application Insights telemetry config is set up and registered with DI in ConfigureServices (so that it picks up all telemetry initializers and processors that are also registered), and therefore only available after it has ConfigureServices has run
• The telemetry config is needed when configuring Serilog
• UseSerilog is configured before ConfigureServices is called

Given Microsoft’s design philosophy, where the composition root of the application is on the level of the IHost, there is now a timing problem in setting up the Serilog logging pipeline. Ideally configuring logging would be about the first thing to do after application startup, which is before building and running the host. However, some logging pipeline components might have dependencies first set up in the composition root.

IMHO a good approach would be to do all fail-safe/isolated logger setup directly after application startup (e.g. configuring a console sink for logging to stdout), and then being able to either add configuration to that same logger, or to derive from it (and update the static instance) in the composition root. A clean API for this might be to add an overload of LoggerConfiguration that would take an existing, already configured Logger. However, I am not sufficiently familiar with the internals of serilog to know whether this can safely be implemented.

It would be great if we can take this discussion forward. (@nblumhardt pardon the indirect nudge 😉)