Review/improve the custom tokenizers for transformers and llamacpp

The models `Transformers` and `LlamaCpp` include some workarounds through the use of an Outlines-defined tokenizer on top of the model's native tokenizer. Those have recently been the cause of bugs and feel a bit hacky. We should look at whether there's a way of improving that part of our codebase to ensure all edge cases are covered and have better maintainability.