Suppress PyTorch Lightning debug output

[shivanraptor]

🐛 Bug

When using the COMET to evaluate sentence pairs, there are debug messages that cannot be suppressed:

To Reproduce

Here is the minimal verifiable example:

import evaluate
import re

def _split_keyword(s):
    regex = r"[+-]?(?:\d+(?:\.\d*)?|\.\d+)%?|[\u4e00-\u9fff]|[A-Za-z]+(?:'[A-Za-z]+)?"
    return [match for match in re.findall(regex, s) if match] 
    
comet = evaluate.load("comet")

yue = 'I am a boy.'
zh = 'I am a good boy.'
ref = 'I am a well behaved boy.'

split_hypo = ' '.join(_split_keyword(zh))
split_ref = ' '.join(_split_keyword(ref))
split_src = ' '.join(_split_keyword(yue))

print(comet.compute(predictions=[split_hypo], references=[split_ref], sources=[split_src]))

which outputs:

Fetching 5 files: 100% (Progress Bar here) 5/5 [00:00<00:00, 638.69it/s]
Encoder model frozen.
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]
{'mean_score': 0.8581651449203491, 'scores': [0.8581651449203491]}

I tried to use a whole bunch of codes to suppress the warnings, but no effect at all:

import logging
warnings.filterwarnings("ignore", category=FutureWarning)
warnings.filterwarnings("ignore", category=UserWarning) 
# Suppress PyTorch Lightning debug messages
logging.getLogger('lightning').setLevel(logging.ERROR) 
logging.getLogger("pytorch_lightning").setLevel(logging.ERROR) 
logging.getLogger("lightning.pytorch.accelerators.cuda").addHandler(logging.NullHandler())
logging.getLogger("lightning.pytorch.utilities.rank_zero").addHandler(logging.NullHandler())
logging.getLogger("pytorch_lightning.accelerators.hpu").addHandler(logging.NullHandler())
def device_info_filter(record):
    return "PU available: " not in record.getMessage()
logging.getLogger("lightning.pytorch.utilities.rank_zero").addFilter(device_info_filter)

Expected behaviour

I expected the code to produce only the score output:

{'mean_score': 0.8581651449203491, 'scores': [0.8581651449203491]}

Screenshots

N/A

Environment

OS: Ubuntu 22.04
pip 25.2
pytorch-lightning 2.5.6
unbabel-comet 2.2.2
nvidia-cuda-runtime 12.4.127

Additional context

N/A

[kennethsible]

@shivanraptor I was able to suppress everything except for the progress bar using:

warnings.filterwarnings('ignore', module='torchmetrics')
warnings.filterwarnings('ignore', module='pytorch_lightning')
logging.getLogger('comet.models').setLevel(logging.WARNING)
logging.getLogger('pytorch_lightning.accelerators.cuda').setLevel(logging.WARNING)
logging.getLogger('pytorch_lightning.utilities.rank_zero').setLevel(logging.WARNING)

However, make sure you put these statements before you import anything from comet

[shivanraptor]

@shivanraptor I was able to suppress everything except for the progress bar using:

warnings.filterwarnings('ignore', module='torchmetrics')
warnings.filterwarnings('ignore', module='pytorch_lightning')
logging.getLogger('comet.models').setLevel(logging.WARNING)
logging.getLogger('pytorch_lightning.accelerators.cuda').setLevel(logging.WARNING)
logging.getLogger('pytorch_lightning.utilities.rank_zero').setLevel(logging.WARNING)
However, make sure you put these statements before you import anything from comet

I missed the comet.models line. Thank you.