Server logging code is very slow

[fafhrd91]

Code log access log generation is very slow.

[ludovic-gasc]

Hi,

Warning: My experimentations below based on my experimentations 1-2 years ago, when I started to use heavily AsyncIO, I was less experimented than now.

About logging, when I've benchmarked, I've remarked that when I use a file or console handler, I'd some performances impacts. The less consuming manner I've found is to use a SyslogHandler that uses UNIX socket, and configure correctly rsyslog to write the log file.

Config example with a logging.ini.
A rsyslog.conf example.

I thought to implement some logging handlers with AsyncIO to reduce the impact.
No time to do that, if somebody else wants to do that, take the candy ;-)

If you find something else, I'm interested in.

[fafhrd91]

this issue is not really related to python logging facility, it is about code in aiohttp that generates log messages for access_log. in my benchmarks turning off logging usually gives 25-30% increase.
and by turning off i mean pass None as access_log parameter to ServerHttpProtocol.

[ludovic-gasc]

You loose 25-30% even if you reduce the log level to ERROR ?

[fafhrd91]

yes, it is not python logging package problem. it is how we generate strings for access log.

[fafhrd91]

problem is here
https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/helpers.py#L216
https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/helpers.py#L251

[fafhrd91]

close by #572

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.