Skip to content

Add tokenizer truncation side option #177

@jon-tow

Description

@jon-tow

🚀 The feature, motivation, and pitch

Some tasks may benefit from truncating on different sides of batched prompts. We should add a configuration option to handle this from a high level as discussed here. Pasting discussion for quick view:

Screenshot 2023-01-09 at 16 28 43

CC @reciprocated and @PhungVanDuy just in case y'all would like to add any extra comments or context.

Alternatives

No response

Additional context

No response

Metadata

Metadata

Labels

contributions welcomePR from open source contributors welcome to solve this issue.feature requestNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions