File based provider?

Hiya,

This seems exactly what a was looking for. Only I have a File sink setup instead of sql. Would it be feasible to implement a file-based provider for serilog-ui?

An excerpt of my log file;

[10:20:51 INF] Sulis python based parser: null {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.Windows.ProcessRunner"}
[10:20:51 INF] Process Sulis python based parser exited with exit code 0 at 19-2-2021 10:20:51 {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.Windows.ProcessRunner"}
[10:20:51 INF] Done running {"Name":"Sulis python based parser","WaitForExit":true,"DurationMilliseconds":882.0070000000001,"TimeOutMilliseconds":0,"ExitedWhenWaitingForTimeout":false,"exitLogInfo":{"ExitCode":0,"ExitTime":"2021-02-19T10:20:51.9647150+01:00"}} {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.Windows.ProcessRunner"}
[10:30:31 INF] Skipping download of remote file /StaticFiles/TemboAnalyticsLogs_20201023.zip, local file ...\Tembo Paper - KAMPEN (NL) - Sulis_P02378_Line1\TDCNL_0045~TemboAnalyticsLogs_20201023.zip was up to date, Local file size: 7116022 vs remote file size: 7116022 {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.HangFireJobs.RemoteDataFilePuller"}
[10:30:31 INF] Skipping download of remote file /StaticFiles/TemboAnalyticsLogs_20201026.zip, local file ...\Tembo Paper - KAMPEN (NL) - Sulis_P02378_Line1\TDCNL_0045~TemboAnalyticsLogs_20201026.zip was up to date, Local file size: 7393688 vs remote file size: 7393688 {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.HangFireJobs.RemoteDataFilePuller"}
[10:30:31 INF] Skipping download of remote file /StaticFiles/TemboAnalyticsLogs_20201027.zip, local file ...\Tembo Paper - KAMPEN (NL) - Sulis_P02378_Line1\TDCNL_0045~TemboAnalyticsLogs_20201027.zip was up to date, Local file size: 7628661 vs remote file size: 7628661 {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.HangFireJobs.RemoteDataFilePuller"}

The logging config is as follows:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information",
      "System.Net.Http.HttpClient": "Information",
      "Hangfire": "Information"
    }
  },
  "Serilog": {
    "Using": [ "Serilog.Sinks.Console" ],
    "WriteTo": [
      {
        "Name": "Console",
        "outPutTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}"
      },
      {
        "Name": "File",
        "Args": {
          "path": "%CUROUTDIR%\\..\\Logs\\RemoteData.WorkerService.txt",
          "rollingInterval": "Day",
          "rollOnFileSizeLimit": true,
          "retainedFileCountLimit": 31,
          "fileSizeLimitBytes": 5242880,
          "outPutTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}"
        }
      }
    ]
  }

I could of course change the template to something that would be easier to parse. I’d just like your opinion if something like this would be feasible and perhaps I take a crack at it.

Adding another provider for sql server for a fancy log viewer seems a bit too much consider the scope of my project. Although perhaps that’s a common solution.

In anyway I want to keep file based logging.

[mo-esmp]

@sommmen you need to implement IDataProvider interface for the file sink. Please check out other providers to see how they are implemented.

There are several issues that need to be considered:

• Paging: how to calculate the total number of logs? (logs are saved in multiple files)
• How to read logs via multiple files (for example, the log file for today contains 5 lines, and the log file last day contains 200 lines and for each page10 records should be displayed. now 5 lines from today’s log file and 5 logs from last day’s log file should be fetched)
• If there were 15 pages and the user clicked on page 7, which file(s) should be opened to read logs?

[mo-esmp]

• First fork the repo
• Checkout dev branch
• Create a feature branch
• Merge feature branch into dev
• Send a pull request