OpenTelemetry Logging with Django

[willianantunes]

Django uses the dictionary LOGGING to configure the logging system (check its documentation). Talking about programmatically configuration, OpenTelemetry Logging Instrumentation can be configured in the manage.py file and gunicorn_config.py file if you are using Gunicorn. If you set LoggingInstrumentor().instrument(tracer_provider=provider, set_logging_format=True) in both files, it won't work properly, because Django will call logging.config.dictConfig(LOGGING) later, and it will overwrite the configuration set by OpenTelemetry. To solve this issue, you can create a class like the following:

import logging

from opentelemetry.sdk._logs import LoggerProvider
from opentelemetry.sdk._logs import LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter

class CustomLoggingHandler(LoggingHandler):
    def __init__(self, level=logging.NOTSET, logger_provider=None) -> None:
        logger_provider = LoggerProvider()
        set_logger_provider(logger_provider)
        logger_provider.add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter()))
        super().__init__(level, logger_provider)

Then you can set the LOGGING dictionary like this:

...
},
"handlers": {
    "console": {
        "class": "logging.StreamHandler",
        "filters": ["request_id", "redact_filter"],
        "formatter": os.getenv("DEFAULT_LOG_FORMATTER", "standard"),
    },
    "otlp": {"class": "otlp.CustomLoggingHandler"},
},
"loggers": {
    "": {"level": os.getenv("ROOT_LOG_LEVEL", "INFO"), "handlers": ["console", "otlp"]},
...

Notice the custom handler named otlp. Remember, according to the documentation, the APIs within opentelemetry.sdk._logs are subject to change. So, it is important to keep an eye on the updates. Libraries I used at the time I posted this comment:

django = "5.1.3"
opentelemetry-sdk = "1.28.1"
opentelemetry-instrumentation-django = "0.49b1"
opentelemetry-instrumentation-psycopg = "0.49b1"
opentelemetry-instrumentation-wsgi = "0.49b1"
opentelemetry-instrumentation-logging = "0.49b1"
opentelemetry-exporter-otlp = "1.28.1"

Sample project.

[overide]

Hi Willian, I'm trying to get it working with Django and facing an issue with the log. I'm using the latest library version, so syntax might have changed. I'd really appreciate it if you could take a look at my discussion thread once - #4792