Q: Can't understand LogLevel settings.

Hi!

This is just a question, or maybe an enhancement for the docs if it’s not just me.

First of all, thank you for this library! Good job! I’ve been using it for about 6 months, had no problem!

Context: Library Version: 3.0.0 (I’ll probably update soon…) Target framework: .NET Core 3.0 (Upgrade coming as well…) It’s a complex ASP.NET Core project, with background tasks etc.

So the one thing I’ve been messing around is setting up my logs to be “just perfect”. I have some Global LogLevel settings:

"Logging": {
    ...
    "LogLevel": {
      "Default": "Trace",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information",
      "NHibernate": "Debug"
    },
    ...
}

Then I have a Console provider which is most probably irrelevant. And lastly I have my File provider settings:

"File": {
      "BasePath": "logs",
      "FileAccessMode": "KeepOpenAndAutoFlush",
      "MaxFileSize": 10485760,
      "CounterFormat": "000",
      "IncludeScopes": true,
      "MaxQueueSize": 1000,
      "Files": [
        {
          "Path": "app-persistence-<counter>.log",
          "MinLevel": {
            "Default": "Trace",
            "NHibernate": "Debug",
            "NHibernate.Type": "Information",
            "NHibernate.Cfg.XmlHbmBinding": "Warning",
            "NHibernate.Cfg.XmlHbmBinding.Binder": "Information",
            "Itg.Persistence": "Trace"
          }
        },
        {
          "Path": "app-<counter>.log",
          "MinLevel": {
            "Default": "Debug",
            "Microsoft": "Warning",
            "Microsoft.AspNetCore.Authentication": "Debug",
            "Microsoft.AspNetCore.Authorization": "Debug",
            "Microsoft.AspNetCore.Server": "Debug",
            "NHibernate": "None",
            "Itg.Persistence": "None",
            "Itg.Services": "Debug",
            "Itg.Services.Shop": "Information",
            "Itg.Mailing": "Information"
          }
        }
      ]
    }

The thing I can’t seem to understand is the following: I’d like my app-persistence-<counter>.log to contain only NHibernate and Itg.Persistence categories, the other one mostly everything else.

On my initial run I had "Default": "None" on my persistence file, which I thought must be right, because I didn’t want anything else to land in there, only the ones I specified (and more specific ones!).

Now based on these, after reading through Microsoft’s logging tutorial, and this repo’s readme, the category "Itg.Persistence.Secondary" should have been written to the persistence file, but it did not! (it’s “under” "Itg.Persistence" which I set to "Trace")

The logging worked, as I could find my entries on the console, but nowhere else. To be exact, this is the line I was looking for: crit: Itg.Persistence.Secondary.FileKeyValueStore[LoadFailed] And to be more exact, this line is logged by a queued background task!

Now I set "Default": "Trace" on my persistence file, and here I thought, now everything should show up. Not just the ones I specified. To my surprise the one I was looking for DID show up ("Itg.Persistence.Secondary") BUT not at all everything… This is the only new category in my persistence log. Which is just about the effect I wanted, but I can’t imagine these settings are any good…

Could you help me out please? What am I missing? I’m so confused… 😦

[adams85]

I think the key momentum here is the blockage of the Itg.Services.* messages. However, I cannot see yet how they can get stuck for minutes. I suspect though it has something to do with the relatively high logging traffic of NHibernate.

Could you do a test where you fire a single(!) request and check the logs after some minutes if the Itg.Services.* entries are still missing? I’d just like to make sure that we’re dealing with just one or two issues here (that is, whether there’s also something regarding configuration reloading or not).

If the test failed (no Itg.Services.* entries after say 5 minutes), then please try the following:

1. Repeat the test described above with unbounded queues (MaxQueueSize set to 0)
2. Repeat the test where FileAccessMode set to KeepOpen and OpenTemporarily. Let’s see if this makes a difference. (There’s an issue with flushing file streams asynchronously in .NET. I ran into perf problems because of this but it’s been solved - theoretically.)
3. Update the file logger lib to the latest version (3.1.1). You can do this safely, I keep to semver (no breaking changes to the public APIs in minor versions). 3.1 comes with some perf-related improvements (e.g. Dataflow was replaced by Channels). Let’s see if this changes anything.

This will take some time of yours, but I’m afraid we cannot do anything else. We have to narrow down the possible causes of your issue to be able to track it down.

[adams85]

Hi!

First of all, thanks for the kind words. 😃

I think your understanding of log level settings are fundamentally correct. There’s one important thing to keep in mind. Log level switches defined at log file level (MinLevel option) can only make the log level switches defined at higher levels (global level or log provider level) stricter. This is mentioned in the docs as well:

[…] this is a second-level filter which can only tighten the rules of the first-level LogLevel filters (as it can only filter messages that hit the logger provider).

This is a limitation of the Microsoft.Extensions.Logging framework I cannot do anything about as it’s out of my control. However, in your case this cannot cause any problems at all (theoretically).

That is, your configuration should work. I put together a simplified imitation of your environment based on your description but I failed to reproduce the issue. It just works as expected.

Please take a look at my attempt and try to figure out if there’s some relevant differences between my simulated and your real environment which I don’t see.