Skip to content

Commit 07e5877

Browse files
scoring update
Signed-off-by: Brian Dellabetta <[email protected]>
1 parent 4160837 commit 07e5877

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

tests/lmeval/configs/vl_w4a16_actorder_weight.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,9 @@ lmeval:
1414
task: mmmu_val_economics
1515
num_fewshot: 0
1616
# dense model achieves 0.4 accuracy
17-
# result is lower (0.366) with
17+
# result is the same (0.366) with
1818
# "quantization_config": CompressedTensorsConfig(run_compressed=False)
1919
# in model_args
2020
metrics:
21-
acc,none: 0.4
21+
acc,none: 0.366
2222
batch_size: 4

0 commit comments

Comments
 (0)