[BUG] Text.append_tokens does not strip control codes #3014
Description
- I've checked docs and closed issues for possible solutions.
- I can't find my issue in the FAQ.
Describe the bug
A string containing control codepoints passed to Text.append_tokens sometimes crashes with IndexError when printed
Related issues: #2284 and #2963
Minimal code example that demonstrates the issue:
>>> from rich import get_console
>>> from rich.text import Text
>>> t = Text().append_tokens([("long text that will be wrapped with a control code \r\n.", "red")])
>>> get_console().print(t, width=40)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/tmp/.venv/lib/python3.10/site-packages/rich/console.py", line 1699, in print
extend(render(renderable, render_options))
File "/tmp/.venv/lib/python3.10/site-packages/rich/console.py", line 1331, in render
for render_output in iter_render:
File "/tmp/.venv/lib/python3.10/site-packages/rich/text.py", line 654, in __rich_console__
lines = self.wrap(
File "/tmp/.venv/lib/python3.10/site-packages/rich/text.py", line 1184, in wrap
new_lines = line.divide(offsets)
File "/tmp/.venv/lib/python3.10/site-packages/rich/text.py", line 1113, in divide
line_start, line_end = line_ranges[end_line_no]
IndexError: list index out of rangePlatform
Click to expand
$ python -m rich.diagnose
╭───────────────────────── <class 'rich.console.Console'> ─────────────────────────╮
│ A high level console interface. │
│ │
│ ╭──────────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=102 ColorSystem.EIGHT_BIT> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ color_system = '256' │
│ encoding = 'utf-8' │
│ file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> │
│ height = 51 │
│ is_alt_screen = False │
│ is_dumb_terminal = False │
│ is_interactive = True │
│ is_jupyter = False │
│ is_terminal = True │
│ legacy_windows = False │
│ no_color = False │
│ options = ConsoleOptions( │
│ size=ConsoleDimensions(width=102, height=51), │
│ legacy_windows=False, │
│ min_width=1, │
│ max_width=102, │
│ is_terminal=True, │
│ encoding='utf-8', │
│ max_height=51, │
│ justify=None, │
│ overflow=None, │
│ no_wrap=False, │
│ highlight=None, │
│ markup=None, │
│ height=None │
│ ) │
│ quiet = False │
│ record = False │
│ safe_box = True │
│ size = ConsoleDimensions(width=102, height=51) │
│ soft_wrap = False │
│ stderr = False │
│ style = None │
│ tab_size = 8 │
│ width = 102 │
╰──────────────────────────────────────────────────────────────────────────────────╯
╭─── <class 'rich._windows.WindowsConsoleFeatures'> ────╮
│ Windows features available. │
│ │
│ ╭───────────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(vt=False, truecolor=False) │ │
│ ╰───────────────────────────────────────────────────╯ │
│ │
│ truecolor = False │
│ vt = False │
╰───────────────────────────────────────────────────────╯
╭────── Environment Variables ───────╮
│ { │
│ 'TERM': 'xterm-256color', │
│ 'COLORTERM': None, │
│ 'CLICOLOR': None, │
│ 'NO_COLOR': None, │
│ 'TERM_PROGRAM': None, │
│ 'COLUMNS': None, │
│ 'LINES': None, │
│ 'JUPYTER_COLUMNS': None, │
│ 'JUPYTER_LINES': None, │
│ 'JPY_PARENT_PID': None, │
│ 'VSCODE_VERBOSE_LOGGING': None │
│ } │
╰────────────────────────────────────╯
platform="Linux"$ pip freeze | grep rich
rich==13.4.2