Debug Logging (Maybe just a n00b issue)

Disclaimer: n00b to Python altogether. This is the first Python app that I’ve ever built.

Trying to add formatted debug logging to the screen. I’ve gotten it to the point where the debug output shows twice. Once with the formatting applied, and once raw.

Scaled down code would be:

import logging

from fastapi import FastAPI

api = FastAPI()

logger = logging.getLogger("api")
logger.setLevel(logging.DEBUG)

fh = logging.FileHandler("QED.log")
fh.setLevel(logging.ERROR)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
ch.setFormatter(formatter)

logger.addHandler(fh)
logger.addHandler(ch)

@api.get("/")
def read_root():
    logger.debug("THIS IS A DEBUG LOG")
    logger.error("SOMETHING WENT VERY VERY WRONG")
    return {"Hello": "World"}

Output to the console is:

2019-06-04 12:44:51,169 - api - DEBUG - THIS IS A DEBUG LOG
DEBUG: THIS IS A DEBUG LOG
2019-06-04 12:44:51,169 - api - ERROR - SOMETHING WENT VERY VERY WRONG
ERROR: SOMETHING WENT VERY VERY WRONG
INFO: ('127.0.0.1', 55229) - "GET / HTTP/1.1" 200

So, two things are happening.

1. Seeing both the formatted message and the raw message.
2. Seeing the ERROR level message in the console. That should only be written to QED.log

Thanks!

[davidrios]

Hi @charliegriefer

You don’t need to set handlers for logging in your FastAPI app, the code can just be:

import logging

from fastapi import FastAPI

api = FastAPI()

logger = logging.getLogger("api")

@api.get("/")
def read_root():
    logger.debug("THIS IS A DEBUG LOG")
    logger.error("SOMETHING WENT VERY VERY WRONG")
    return {"Hello": "World"}

Then you set the logging level when starting uvicorn, like:

uvicorn qed.api:api --reload --log-level debug

When you run your application in a server, if it is Linux with systemd, it’ll take care of saving the log to the journal and syslog, as well as timestamping it, then you can redirect to another file if you want. But you don’t need to worry about that when developing, just at deployment time.

[euri10]

ok so basically I’m using this, in a applog package

import logging.config
import os

import yaml


def read_logging_config(default_path="logging.yml", env_key="LOG_CFG"):
    path = default_path
    value = os.getenv(env_key, None)
    if value:
        path = value
    if os.path.exists(path):
        with open(path, "rt") as f:
            logging_config = yaml.safe_load(f.read())
        return logging_config
    else:
        return None


def setup_logging(logging_config, default_level=logging.INFO):
    if logging_config:
        logging.config.dictConfig(logging_config)
    else:
        logging.basicConfig(level=default_level)

you use it wherever your entrypint is this way

from applog.utils import read_logging_config, setup_logging

logconfig_dict = read_logging_config("applog/logging.yml")
setup_logging(logconfig_dict)

should you need, you can pass env another file than the default logging.yml for your dev / prod environements

logging.yml looks like this:

version: 1
disable_existing_loggers: False
formatters:
    simple:
        format: "%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(message)s"
handlers:
    console:
        class: logging.StreamHandler
        level: DEBUG
        formatter: simple
        stream: ext://sys.stdout
    info_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: INFO
        formatter: simple
        filename: applog/info.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8
    error_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: ERROR
        formatter: simple
        filename: applog/errors.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8
loggers:
    uvicorn:
        level: INFO
        handlers: [console]
        propagate: no
    gunicorn:
        level: INFO
        handlers: [console]
        propagate: no
root:
    level: INFO
    handlers: [console]