Text input support

[ickshonpe]

Objective

Add support for text inputs using cosmic text's Editor API.

Solution

Two new UI widgets, TextBox and TextField. TextBox is for full multi-line text editing. TextField is for single line text editing and supports numerical and password modes.

There are three new modules:

• bevy_text::input: Contains the text buffer, undo history and layout components, handles their updates and dispatches TextInputEvents.
• bevy_ui::widgets::text_box: Contains common shared UI text input systems and components and the TextBox input widget implementation.
• bevy_ui::widgets::text_field: Contains the TextField specific components and systems.

Rendering is handled by extract_text_input_nodes in the bevy_ui_render crate.

TextBox and TextField have their own plugin TextInputPlugin that isn't included by default because they require the InputDispatchPlugin to work.

Not very satisfied with the implementation but needed to stop wasting time fussing around with it. There are too many features for a single PR and some parts still feel a bit chaotic and over complicated. In particular state management is a bit all over the place. Each text input widget has a queue of TextInputActions that are applied to the cosmic-text buffer in PostUpdate after layout updates. There is a TextInputValue component that, if present, is updated each frame with the current text from the buffer. Inserting a new TextInputValue overwrites the current contents of the buffer. Maybe it needs a Callback somewhere as well, possibly replacing the TextInputValue component, I'm not sure.

The widget modules were put in the bevy_ui crate rather than bevy_core_widgets or bevy_feathers. Was very indecisive about this but bevy_core_widgets is for widgets with no inherent styling and the text input widgets are inherently styled because of the fonts.

Features

• Copy, cut and paste from system clipboard.
• Password entry mode.
• Mouse wheel scrolling.
• Numerical entry modes.
• Single line input.
• Multiline inputs.
• Max characters limit.
• Propagated events that are emitted on submission, changes to the text, and invalid inputs.
• Custom filters to disallow invalid inputs.
• Horizontal and vertical scrolling.
• Text selection.
• Drag to select.
• Double-click to select words, triple-click to select lines.
• Full Undo and Redo support.
• Indent/Unindent.
• Insert and overwrite edit modes.
• Automatically display a prompt when the input is empty.
• Clear on submit.
• Set height automatically to fit a given number of lines.

Bugs

• Cursor is always left-justified when an input is empty.
• Password mode layout can get out of sync with variable width fonts.
• Mouse wheel scrolling ignores pixel values.
• Horizontal scrolling seems inconsistant sometimes.

Not supported yet

• IME (there's a PR for this on the text input crate that might be upstreamable).
• VIM modes.
• Max lines limit.
• Drag to change support for numerical input fields.
• Input field previous submission history.
• Scrollbar support.
• Automatically add scroll bars on overflow.
• Select and copy from regular Text entities.
• bevy_2d input using text2d
• Curved corner support for selection rects
• Shift and click to select.

Testing

Four examples:

• text_input
• line_input_node
• password_input
• text_box

The text_input example is the most complete, the others are rougher ones I was using for testing. Needs an example to make sure that the widgets update correctly and don't get stuck in an invalid state after multiple changes to the font sizes, line height, visible lines, target size etc..

Showcase

I'll complete the migration guides and release notes a bit later, going to take a break for a few hours.

[ickshonpe]

• Most of the text input libraries don't support "virtual keyboard" entry which means they don't work on mobile/tablet web builds. For example see Keyboard not displayed on mobile WASM builds rparrett/bevy_simple_text_input#75 for some discussion. I wasn't able to build for web but looking through the code I think the same is true here? This one seems like a very difficult problem to solve although supposedly EGUI does it via a hidden input.

It should be trivial to create a basic virtual keyboard by spawning a button for each key and on button presses sending the corresponding TextInputActions to the text input entity with InputFocus. I'm not sure what would be involved in hooking it up to a builtin OS keyboard implementation API but it shouldn't be prohibitively difficult.

[viridia]

TextBox is for full multi-line text editing. TextField is for single line text editing and supports numerical and password modes.

On the web, a multi-line text input is called a "textarea" - I think that might be a better name, or at least one that users will be more familiar with. The problem I have with the word "box" is that it doesn't immediately suggest multi-line-ness to me (since both multi-line and single-line inputs are technically "boxes" as well as "fields"). (And, yes, you can also argue that the word "area" applies to both as well - but the long-established convention in HTML is that area is multi-line.)

[viridia]

I patched in the PR to my local workspace and ran the examples. One of the first things I did was to try and paste in some arabic text (just go to any arabic website and copy) but then I realized we don't have clipboard support yet. Doh!

Note that even if we did have clipboard support, I wouldn't expect non-latin scripts to work out of the box unless we used a font with support for that language.

[alice-i-cecile]

I want to merge #20336 first in time for 0.17, then rebase and land this for the start of Bevy 0.18.

[ickshonpe]

I patched in the PR to my local workspace and ran the examples. One of the first things I did was to try and paste in some arabic text (just go to any arabic website and copy) but then I realized we don't have clipboard support yet. Doh!

Note that even if we did have clipboard support, I wouldn't expect non-latin scripts to work out of the box unless we used a font with support for that language.

Since this missed the 0.17 cutoff, I might as well merge in the bevy_clipboard crate PR now and hook it up as it's going to be super helpful for testing.

[viridia]

All right. There is another advantage (for me) to doing the widgets post-0.17, which is that by then we should have a preliminary version of BSN in-tree. Since all of the feathers widgets are going to be converted to BSN, if I am going to write a feathers text input based on this I'd rather it be BSN from the start.

I attempted to merge cart's BSN branch with this PR, and the merge mostly went OK, but there are some sticking points as cart's branch is about two weeks old, and there are some other conflicts:

crates/bevy_text/src/input.rs:231:19
    |
231 |             font: Default::default(),
    |                   ^^^^^^^^^^^^^^^^^^ the trait `std::default::Default` is not implemented for `bevy_asset::Handle<font::Font>`

I did run all the examples, and things look really good overall. I haven't yet gone through the PR line-by-line.

[viridia]

BTW, a site I discovered today is "arabic lorem ipsum": https://www.loremipzum.com/ar/

[ickshonpe]

TextBox is for full multi-line text editing. TextField is for single line text editing and supports numerical and password modes.

On the web, a multi-line text input is called a "textarea" - I think that might be a better name, or at least one that users will be more familiar with. The problem I have with the word "box" is that it doesn't immediately suggest multi-line-ness to me (since both multi-line and single-line inputs are technically "boxes" as well as "fields"). (And, yes, you can also argue that the word "area" applies to both as well - but the long-established convention in HTML is that area is multi-line.)

I'm not in love with the name "TextBox" either, I've changed it multiple times already. "TextArea" seems like it would be fine and it makes sense to pick something familiar.

[ickshonpe]

I patched in the PR to my local workspace and ran the examples. One of the first things I did was to try and paste in some arabic text (just go to any arabic website and copy) but then I realized we don't have clipboard support yet. Doh!

Note that even if we did have clipboard support, I wouldn't expect non-latin scripts to work out of the box unless we used a font with support for that language.

You can use the TextInputValue component to inject text into a text input buffer:

commands
        .spawn((
            TextField::default(),
            TabIndex(0),
            TextColor(RED.into()),
            TextInputValue::new("ماذا يعني لوريم إيبسوم الم؟"),
            TextFont {
                font: asset_server.load("fonts/NotoNaskhArabic-Medium.ttf"),
                font_size: 30.,
                line_height: LineHeight::Px(50.),
                ..Default::default()
            },
            Node {
                width: Val::Percent(50.),
                ..default()
            },
            BackgroundColor(NAVY.into()),
        ));

And then you can edit the arabic text and mix it up with english glyphs:

But it's not correct, cursor navigation in particular doesn't seem to make sense. I'm not certain what the correct behaviour should be though. I'll try and find an arabic text input on the web that I can do a comparison with.

[ickshonpe]

Added the system clipboard support just now, seems to work but I'll clean it up and test it properly tomorrow.

[viridia]

it's not correct, cursor navigation in particular doesn't seem to make sense.

I'm less concerned right now that we get the correct behavior, as long as some Bevy developer that is familiar with these input methods can hack on it without having to re-invent the whole framework from scratch.

I'll ask on Discord to see if anyone knows about this.

[viridia]

There are some subtle issues with selection rects.

For the multi-line widget, when I select a line ending in either a soft or hard break (that is, either a space or newline), the selection rect does not include the whitespace character. This creates an ambiguous situation whereby the user cannot tell whether the selection includes the line ending or not.

It used to be the case that for most desktop operating systems, when a selection included whitespace at the end of a line, the selection rect would be extended all the way to the right margin of the text input widget. However, this rule is not consistently enforced: VSCode and Chrome don't do this, but Scrivener (which is the only word-processor-like app I currently have installed) does.

For bidirectional text, the example video posted in the discord channel makes me think that perhaps we're not properly handling discontiguous selection rects in the single-line case. If I have a run of text consisting of 10 characters of LTR script, followed by 10 characters of RTL script, and if the selection extends from character index 5 to 15, then there should be two separate selection rects:

L L L L L [L L L L L ] R R R R R [R R R R R]

In the single-line case, there can be at most 3 separate selection rects.

I am OK if these issues are addressed in a follow-up PR or even a point release, as they are not critical for most users.