Unable to bypass psutil (requesting feature to force constant mode)

[laura-reumont]

Hello, thank you for your work on CodeCarbon!

I'm currently using CodeCarbon to estimate energy usage in a machine learning setting, where I track short training iterations (e.g forward pass, backward pass, etc).

I'm observing a significant overhead when using CodeCarbon v3.0.2:

• Full training time without CodeCarbon: 11 seconds
• Full training time with CodeCarbon v3.0.2: 1 minute 55 seconds

I'm working in an environment without RAPL, PowerGadget, or PowerMetrics. I identified the overhead as coming from repeated calls to psutil.cpu_percent(interval=0.5), which takes 0.5 seconds per call. In contrast, when I use CodeCarbon v2.8.4 (before psutil support) it incurrs negligible overhead (<1s).

I am therefore trying to bypass using psutil (i.e. bypassing CPU Load mode). I attempted to force CodeCarbon into constant mode by uninstalling psutil, expecting it to fall back as it does for unavailable backends like RAPL or PowerGadget. However, this leads to a ModuleNotFoundError due to an unconditional import of psutil in core/cpu.py.

Looking into the source, I saw that CodeCarbon uses the following health check:

is_psutil_available():
    try:
        nice = psutil.cpu_times().nice
        if nice > 0.0001:
            return True
        else:
            logger.debug(
                f"is_psutil_available() : psutil.cpu_times().nice is too small : {nice} !"
            )
            return False
    except Exception as e:
        logger.debug(
            "Not using the psutil interface, an exception occurred while instantiating "
            + f"psutil.cpu_times : {e}",
        )
        return False

This differs for example from is_powergadget_available() which explicitly looks for the Power Gadget executable.

Would it be possible to:

• Add a parameter (e.g. force_mode_constant=True) to explicitly force constant mode, similar to force_mode_cpu_load
• Or change the psutil import logic to allow it to fail gracefully when the module is missing, rather than assuming it will always be installed

Alternatively, do you know of a way to make the call to is_psutil_available() fail?

Thanks again for maintaining this project!

[laura-reumont]

Update: I have found a way to make is_psutil_available() return False, hence allowing CodeCarbon to fall back to constant mode. You can use a monkey patch:

import codecarbon.core.cpu
codecarbon.core.cpu.is_psutil_available = lambda: False

This fixes my overhead problem. However, it is not a clean fix.

[benoit-cty]

Hello,
Thank you very much for reporting this !
How do you use CodeCarbon, is it in task mode with many very small tasks ?
Because the psutil.cpu_percent(interval=0.5) is supposed to be called only every 15 seconds ( see https://github.com/mlco2/codecarbon/blob/master/codecarbon/emissions_tracker.py#L272 )
The overhead suggest psutil is called ~200 times.

[laura-reumont]

Hi, thanks for your response!
Yes, I'm using trackers in task mode with small tasks (<1s). Each step within a training iteration (forward pass, backward pass, optimisation step, etc) is one task. If I understood the source code correctly, although the main scheduler is disabled during task mode, psutil.cpu_percent(interval=0.5) is called with each start_task() and stop_task() call. Doing a quick back-of-the-envelope calculation, I have 25 training iterations in my example, with 4 tasks per iteration hence 8 calls to psutil per iteration. 8 * 25 = 200 total calls, like you estimated. 200 * 0.5 sec = 100sec = 01:40min, hence the overhead I noticed from ~10s without CodeCarbon to ~01:50min with CodeCarbon. This is why I tried to avoid using pustil, attempting to fall back on constant mode instead of load mode.

[benoit-cty]

To be honest, I don't think you will get good result this way : the measure is done before and after the task. so your CPU load will be low.
If you have an Nvidia GPU or RAPL it will works well as we read their energy counter.

Maybe we could set the interval to 0 as it will give the mean since the previous call if I understand it well : giampaolo/psutil#878

[laura-reumont]

True, CPU load mode definitely isn't the way to go in my case. The issue I had is that I wasn't able to circumvent load mode to use constant mode instead (while I wait for RAPL to be installed on our systems), since psutil is imported in cpu.py. I'm not sure if psutil_is_available() can ever return False given that CodeCarbon crashes before that point if psutil is not installed in the environment. So it seems constant mode can not occur.

Re: giampaolo/psutil#878

cpu_percent() can already be used in a non-blocking manner: just specify interval=0 or interval=None (which is the default BTW)

I wasn't aware that we could specify the interval without modifying the source code.

[benoit-cty]

No you can't do it, we have to change this in the code of CodeCarbon and evaluate if results are better.

[bnestor]

Another important use-case is for monitoring inference. I am trying to monitor individual users' CO2 emissions on asynchronous server calls. Unfortunately that means running several <1s jobs and summing the results afterwards. For others interest, I measured the time to do the following:

timeit for 10 iterations on a simple operation

# with no modification to psutil
Decorator time: 53.2206 seconds # decorator on endpointHandler.__call__()
Tracker time: 10.0306 seconds # tracker initialized in endpointHandler.__init__()
Tracker without CodeCarbon time: 0.0000 seconds

# with modification to psutil
Decorator time: 43.0719 seconds # decorator on endpointHandler.__call__()
Tracker time: 0.0052 seconds # tracker initialized in endpointHandler.__init__()
Tracker without CodeCarbon time: 0.0000 seconds