Token accuracy counting method

[AngledLuffa]

When you measured tag accuracy, were you counting B-PER and I-PER as the same, or were you treating them as scoring differently? Measured by entity or by token? In other words, if the correct answer was

John_B-PER Bauer_I-PER

and your tool only labeled

Bauer_B-PER

was this

• one wrong entity
• one correct token, one wrong token
• or two wrong tokens?

Thanks again!

[suralk]

1. When you measured tag accuracy, were you counting B-PER and I-PER as the same? Yes. We didn't use different scores for them, and treated them as the same 2. Measured by entity or by token? Measured as tokens 3. If the correct answer was John_B-PER Bauer_I-PER and your tool is only labeled Bauer_B-PER? If our model output is Bauer_B-PER, then that's two wrong tokens.

…

On Thu, 29 May 2025 at 09:25, John Bauer ***@***.***> wrote: *AngledLuffa* created an issue (suralk/multiNER#2) <#2> When you measured tag accuracy, were you counting B-PER and I-PER as the same, or were you treating them as scoring differently? Measured by entity or by token? In other words, if the correct answer was John_B-PER Bauer_I-PER and your tool only labeled Bauer_B-PER was this - one wrong entity - one correct token, one wrong token - or two wrong tokens? Thanks again! — Reply to this email directly, view it on GitHub <#2>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2X7KS53NPP452G7ISBLMD3AYSWHAVCNFSM6AAAAAB6D5RO6SVHI2DSMVQWIX3LMV43ASLTON2WKOZTGA4TQNJZG4ZTEMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[AngledLuffa]

1. When you measured tag accuracy, were you counting B-PER and I-PER as the
   same?
   Yes. We didn't use different scores for them, and treated them as the
   same

and

3. If the correct answer was John_B-PER Bauer_I-PER and your tool is only
   labeled Bauer_B-PER?
   If our model output is Bauer_B-PER, then that's two wrong tokens.

These seem like contradictory statements, but perhaps the meaning is that B-PER is its own token label, and I-PER is its own token label, and any confusion between B-PER and I-PER are treated as errors?

Thanks for the clarification!