Stackdriver - Structured logging

Tinylog 2.3.1

I want to define a console log to push logs from k8s to google stackdriver with structured logging json format.

https://cloud.google.com/logging/docs/structured-logging

I have defined console log format to output json to console but curly braces are not being escaped properly

#console
writer2=console
writer2.format=\\{\"timestamp\":\"{date:yyyy-MM-dd HH:mm:ss.SSS}\",\"severity\":\"{level}\",\"threadId\":\"{thread}\",\"file\":\"{file}:{line}\",\"message\":\"{message}\"\\}

Actual: \"timestamp":"2021-06-01 22:01:51.137","severity":"DEBUG","threadId":"main","file":"HikariConfig.java:1098","message":"test message"\ Expected: {"timestamp":"2021-06-01 22:01:51.137","severity":"DEBUG","threadId":"main","file":"HikariConfig.java:1098","message":"test message"}

[dbacinski]

I ended up creating console json writer based on JsonWriter:

#console
writer2=console json
writer2.level=debug
writer2.field.severity=level
writer2.field.timestamp={date:yyyy-MM-dd'T'HH:mm:ss.SSSXXX}
writer2.field.thread=thread
writer2.field.ref={file}:{line}
writer2.field.file=file
writer2.field.class=class
writer2.field.message=message

ConsoleJsonWriter:

package com.example.logging

import org.tinylog.Level
import org.tinylog.core.LogEntry
import org.tinylog.core.LogEntryValue
import org.tinylog.pattern.FormatPatternParser
import org.tinylog.pattern.Token
import org.tinylog.writers.Writer
import java.io.IOException
import java.util.*
import kotlin.collections.HashMap

class ConsoleJsonWriter(properties: Map<String?, String?>) : Writer {
    private var builder: StringBuilder? = java.lang.StringBuilder(BUFFER_SIZE)
    private val jsonProperties: Map<String, Token> = createTokens(properties)
    private var errorLevel: Level = Level.ERROR

    override fun getRequiredLogEntryValues(): MutableCollection<LogEntryValue> {
        val values: MutableCollection<LogEntryValue> = EnumSet.noneOf(LogEntryValue::class.java)
        for (token in jsonProperties.values) {
            values.addAll(token.requiredLogEntryValues)
        }
        return values
    }

    override fun write(logEntry: LogEntry) {
        val builder: StringBuilder = if (this.builder == null) {
            StringBuilder()
        } else {
            val builder = this.builder!!
            builder.setLength(0)
            builder
        }
        addJsonObject(logEntry, builder)
        val data = builder.toString()
        if (logEntry.level.ordinal < errorLevel.ordinal) {
            print("$data$NEW_LINE")
        } else {
            System.err.print("$data$NEW_LINE")
        }
    }

    override fun flush() {
    }

    override fun close() {
    }

    /**
     * Prepares and adds a Json Object. Also escapes special characters.
     *
     * @param logEntry LogEntry with information for token
     * @param builder  Target for the created the JSON object
     */
    private fun addJsonObject(logEntry: LogEntry, builder: java.lang.StringBuilder) {
        builder.append("{")
        val tokenEntries = jsonProperties.values.toTypedArray()
        val fields = jsonProperties.keys.toTypedArray()
        for (i in tokenEntries.indices) {
            builder.append("\"").append(fields[i]).append("\":\"")
            val start = builder.length
            val token = tokenEntries[i]
            token.render(logEntry, builder)
            escapeCharacter("\\", "\\\\", builder, start)
            escapeCharacter("\"", "\\\"", builder, start)
            escapeCharacter(NEW_LINE, "\\n", builder, start)
            escapeCharacter("\t", "\\t", builder, start)
            escapeCharacter("\b", "\\b", builder, start)
            escapeCharacter("\u000C", "\\f", builder, start)
            escapeCharacter("\n", "\\n", builder, start)
            escapeCharacter("\r", "\\r", builder, start)
            builder.append("\"")
            builder.append(",")
        }
        builder.deleteCharAt(builder.lastIndexOf(","));
        builder.append("}")
    }

    private fun escapeCharacter(character: String, escapeWith: String, stringBuilder: java.lang.StringBuilder,
                                startIndex: Int) {
        var index = stringBuilder.indexOf(character, startIndex)
        while (index != -1) {
            stringBuilder.replace(index, index + character.length, escapeWith)
            index = stringBuilder.indexOf(character, index + escapeWith.length)
        }
    }

    companion object {
        private val NEW_LINE = System.getProperty("line.separator")
        private const val BUFFER_SIZE = 1024
        private const val FIELD_PREFIX = "field."

        private fun createTokens(properties: Map<String?, String?>): Map<String, Token> {
            val parser = FormatPatternParser(properties["exception"])
            val tokens: MutableMap<String, Token> = HashMap()
            for ((key, value) in properties) {
                if (key!!.toLowerCase(Locale.ROOT).startsWith(FIELD_PREFIX)) {
                    tokens[key.substring(FIELD_PREFIX.length)] = parser.parse(value)
                }
            }
            return tokens
        }
    }
}

result:

{"severity":"DEBUG","ref":"HikariConfig.java:1098","file":"HikariConfig.java","thread":"main","message":"readOnly........................false","class":"com.zaxxer.hikari.HikariConfig","timestamp":"2021-06-02T16:37:33.979+02:00"}

only annoying thing is that it does not keep attributes order from config and timestamp is going last but it doesn’t matter for Stackdriver

[pmwmedia]

These are the characters that tinylog’s JSON writer escapes: https://github.com/tinylog-org/tinylog/blob/v2.4/tinylog-impl/src/main/java/org/tinylog/writers/JsonWriter.java#L147