Is rendering the relevance as percentage the right choice?

[marcominerva]

In the SearchClient, when rendering a Fact template, there is the following instructions:

kernel-memory/service/Core/Search/SearchClient.cs

Line 273 in 99d25a7

relevance: relevance.ToString("P1", CultureInfo.CurrentCulture),

So, the relevance is formatted as a percentage. I'm wondering if it is correct, given that its value can significantly change between models or even model versions.

[dluc]

The value is in most cases a cosine similarity, and it is used to sort facts from more similar to less similar. It is injected in the prompt to give a hint to the model about what is more relevant. The actual value is not important.

[marcominerva]

My doubt is about the percentage format. As the actual value is not important, why doesn't leave it unchanged?

[dluc]

The reason why I went with % is that it carries more semantic meaning compared to a number, considering also that LLMs are trained on human content. For instance, saying "Its' 100% relevant" is more common (and meaningful) than saying "It's 1.0 relevant".
If one wants to customize the prompt, my suggestion would be to run tests with something like Ragas, DeepEval, Deepchecks, ChainForge, LLM-rAG-eval, etc