Repository navigation
logging: Logger.setLevel() and logging.disable() are O(number of loggers) #160936
Description
Bug report
Every Logger.setLevel() and logging.disable() call goes through Manager._clear_cache(), which walks the whole loggerDict and clears each logger's isEnabledFor cache, even the ones that never cached anything. So setting the level on N loggers one by one is quadratic. That shows up in apps or test suites with thousands of loggers (one per module/plugin/connection) that get configured individually.
import logging, time
for n in (1000, 2000, 4000, 8000):
logging.root.manager.loggerDict.clear()
loggers = [logging.getLogger(f"app.mod{i}") for i in range(n)]
for lg in loggers:
lg.isEnabledFor(logging.DEBUG)
t = time.perf_counter()
for lg in loggers:
lg.setLevel(logging.INFO)
print(f"{n:5d} loggers: {time.perf_counter() - t:.3f} s")On 3.14.7 this gives about 0.23 s at 2000 loggers, 0.89 s at 4000 and 4.4 s at 8000 (each doubling is about 4x).
A fix is to have the manager remember which loggers actually have something in their cache (a WeakSet filled on the isEnabledFor cache-miss path, which already holds the lock) and only clear those. The cache-hit path doesn't change. I have a patch ready and can open a PR.
CPython versions tested on:
3.14, CPython main branch
Operating systems tested on:
Linux
Activity
aisk commentedon Oct 30, 2026
Using thousands of loggers is not a common case in real world code, and
setLevel()/disable()is not called very often.So I'm curious, did you run into a real performance problem with these? If not, I doubt this optimization is worth it.
pythongh-160936: Clear all cached loggers unconditionally in _clear_c…
Ar-maan05 commentedon Nov 1, 2026
Good question! Not from an app of my own, I spotted it reading the code. It does match something people really write though: looping over
loggerDictand callingsetLevel()on each logger, like metavoice-src and ktrain do. It's about 75 ms at 2000 loggers and 1.5 s at 8000 on 3.14, so it only really bites big setups.The PR is small and doesn't touch the cache-hit path, and it now also fixes a stale cache bug on standalone loggers (
Logger("x").setLevel(ERROR)leavesisEnabledFor(DEBUG)True), so I think it's worth having even if the speedup is niche.pythongh-160936: Check cache state before and after in the new loggin…
pythongh-160936: Clear the logger cache after the new logging tests