Fail-over to a second server if the first is unreachable

If the attempt to send events to the primary Seq server fails, the sink could be configured to try a secondary Seq server, or more conveniently, a Seq Forwarder that will persist the events and transmit them back to the primary server once it’s available.

For example:

Log.Logger = new LoggerConfiguration()
    .WriteTo.Seq(...
        secondaryServerUrl: "https://some-seq-forwarder",
        secondaryApiKey: "789078907890789")
    .CreateLogger();

Some questions:

• Would this happen immediately, or would it be triggered after some number of failed retries?
• An alternative might be to accept an array of server/API key pairs, however this won’t be as friendly to use from app settings. Is more than one secondary server necessary?
• Assuming that 4xx errors would not fail-over - certainly this would make sense for 400, but what about 404, 401?
• Could the sink somehow signal that it was in failed-over mode, e.g. by adding an event or property, or would a prolonged fail-over be detected by other means (perhaps by attaching a property through the secondary forwarder?)

[tsimbalar]

Just some more thoughts …

Maybe a more “general” way would be to wrap it all , something like this :

Log.Logger = new LoggerConfiguration()
    .WriteTo.Try(w => w.Seq(primary, ), failoverOptions)
          .ThenTry(w => w.Seq(secondary, ), failoverOptions)
          .FallbackTo(w => w.File(secondary, ))
    .CreateLogger();

The idea being that the w in Try(w => w), ThenTry(w => w) is a special kind of SinkConfiguration that specific sinks could light up when they support fallback … (i.e. they provide a way to detect that the destination is currently unavailable).

And then there’s the question of AuditTo … should it also support failover and what would that look like ?

[nblumhardt]

Hi @kevingates!

IBatchedLogEventSink now exists in Serilog.Sinks.PeriodicBatching, and I think we’re happy enough with the shape of it that it could move to Serilog if needed:

https://github.com/serilog/serilog-sinks-periodicbatching/blob/dev/src/Serilog.Sinks.PeriodicBatching/Sinks/PeriodicBatching/IBatchedLogEventSink.cs

There’s no official FallbackSink out there that I know of, but I suspect that one could now be hacked together as a proof-of-concept for sinks that already support IBatchedLogEventSink (like the Seq one, email, and several others) using LoggerSinkConfiguration.Wrap(), which provides a sneaky way to get at the ILogEventSink behind friendly WriteTo configuration methods, and a dose of good-old private reflection to pick up PeriodicBatchingSink’s _batchedLogEventSink field:

https://github.com/serilog/serilog-sinks-periodicbatching/blob/dev/src/Serilog.Sinks.PeriodicBatching/Sinks/PeriodicBatching/PeriodicBatchingSink.cs#L45

A PoC that demonstrated the API and mechanics (either as a gist or example nupkg) would help zoom in on any potential issues, illustrate the API possibilities, and might be enough for us to open up some minimal extension points to make this all possible (either via Serilog.Sinks.PeriodicBatching, or with some core APIs).

HTH!