Merge pull request #788 from o-sdn-o/gui-bridge

Make vt-input-mode mouse reporting better

Author: o-sdn-o

doc/vt-input-mode.md

@@ -31,7 +31,7 @@ Existing approaches have the following drawbacks:
 
 ## Conventions
 
-- We use HEX-form of the uint32 (IEEE-754 32-bit binary float, Little-Endian) for the floating point value representation.
+- We use HEX-form of the uint32 for the 32-bit floating point value representation (IEEE-754 32-bit binary float, Little-Endian).
 - Space characters are not used in sequence payloads and are only used for readability of the description.
 - //todo: keyboard only: All unescaped symbols outside of this protocol should be treated as clipboard pasted data.
 
@@ -63,17 +63,15 @@ Sources      | Events to track
 `"clipboard"`| Clipboard.
 `"window"`   | Window size and selection.
 `"system"`   | System signals.
-`""`         | Empty string to set all event reporting off.
+`""`         | Set event reporting off.
 
 /todo: keyboard only: Note: By enabling `vt-input-mode`, all current terminal modes are automatically saved (to be restored on exit) and switched to something like "raw" mode, in which input is available character by character, echoing is disabled, and all special processing of terminal input and output characters is disabled (except for `LF` to `CR+LF` conversion).
 
 ### Event format
 
-The event signaling also uses APC `ESC _ <payload> ESC \` with an event-specific payload syntax.
-
-The payload consists of a list of attributes in the following format:
+The event signaling also uses APC `ESC _ <payload> ESC \` vt-sequences with the following payload format:
 ```
-<attr>=<val>,...,<val>; ...; <attr>=<val>,...,<val>
+<attr>=<val>,...,<val>;...;<attr>=<val>,...,<val>
 ```
 
 Field             | Descriprtion
@@ -85,15 +83,15 @@ Field             | Descriprtion
 
 - Keyboard
   ```
-  ESC _ event=keyboard ; id=0 ; kbmods=<KeyMods> ; keyid=<KeyId> ; pressed=<KeyDown> ; scancode=<ScanCode> ; id_chord=<HexFormString> ; ch_chord=<HexFormString> ; sc_chord=<HexFormString> ; cluster=<C0>,...,<Cn> ESC \
+  ESC _ event=keyboard ; id=<ID> ; kbmods=<KeyMods> ; keyid=<KeyId> ; pressed=<KeyDown> ; scancode=<ScanCode> ; id_chord=<HexFormString> ; ch_chord=<HexFormString> ; sc_chord=<HexFormString> ; cluster=<C0>,...,<Cn> ESC \
   ```
 - Mouse
   ```
-  ESC _ event=mouse ; id=0 ; kbmods=<KeyMods> ; coor=<X>,<Y> ; buttons=<ButtonState> ; scroll=<DeltaX>,<DeltaY> ; finescroll=<DeltaX>,<DeltaY> ESC \
+  ESC _ event=mouse ; id=<ID> ; kbmods=<KeyMods> ; coor=<X>,<Y> ; buttons=<ButtonState> ; iscroll=<DeltaX>,<DeltaY> ; fscroll=<DeltaX>,<DeltaY> ESC \
   ```
 - Focus
   ```
-  ESC _ event=focus ; id=0 ; state=<FocusState> ESC \
+  ESC _ event=focus ; id=<ID> ; state=<FocusState> ESC \
   ```
 - Format
   ```
@@ -102,7 +100,7 @@ Field             | Descriprtion
 //todo Textinput, Text, IME or Input for IME preview etc
 - Clipboard
   ```
-  ESC _ event=clipboard ; id=0 ; format=<ClipFormat> ; security=<SecLevel> ; data=<Data> ESC \
+  ESC _ event=clipboard ; id=<ID> ; format=<ClipFormat> ; security=<SecLevel> ; data=<Data> ESC \
   ```
 - Window
   ```
@@ -116,15 +114,15 @@ Field             | Descriprtion
 ### Keyboard
 
 ```
-ESC _ event=keyboard ; id=0 ; kbmods=<KeyMods> ; keyid=<KeyId> ; pressed=<KeyDown> ; scancode=<ScanCode> ; id_chord=<HexFormString> ; ch_chord=<HexFormString> ; sc_chord=<HexFormString> ; cluster=<C0>,...,<Cn> ESC \
+ESC _ event=keyboard ; id=<ID> ; kbmods=<KeyMods> ; keyid=<KeyId> ; pressed=<KeyDown> ; scancode=<ScanCode> ; id_chord=<HexFormString> ; ch_chord=<HexFormString> ; sc_chord=<HexFormString> ; cluster=<C0>,...,<Cn> ESC \
 ```
 
 > Q: Do we need to track scancode chord? `scanchord=<Code0>,...,<CodeN>`?
 
 Attribute                     | Description
 ------------------------------|------------
-`id=0`                        | Device group id.
-`kbmods=<KeyMods>`            | Keyboard modifiers.
+`id=<ID>`                     | Device group id (unsigned integer value).
+`kbmods=<KeyMods>`            | Keyboard modifiers bit field.
 `keyid=<KeyId>`               | Physical key ID.
 `pressed=<KeyDown>`           | Key state:<br>\<KeyDown\>=1 - Pressed.<br>\<KeyDown\>=0 - Released.
 `scancode=<ScanCode>`         | Scan code.
@@ -135,7 +133,7 @@ Attribute                     | Description
 
 In response to the activation of `keyboard` tracking, the application receives a vt-sequence containing keyboard modifiers state:
 ```
-ESC _ event=keyboard ; id=0 ; kbmods=<KeyMods> ESC \
+ESC _ event=keyboard ; id=<ID> ; kbmods=<KeyMods> ESC \
 ```
 
 The full sequence is fired after every key press and key release. The sequence can contain a string generated by a keystroke as a set of codepoints: `C0 + ... + Cn`. ~~The string can be fragmented and delivered by multiple consecutive events.~~
@@ -362,17 +360,17 @@ Key ID | Name               | Generic Name       | Scan Code | Notes
 ### Mouse
 
 ```
-ESC _ event=mouse ; id=0 ; kbmods=<KeyMods> ; coor=<X>,<Y> ; buttons=<ButtonState> ; scroll=<DeltaX>,<DeltaY> ; finescroll=<DeltaX>,<DeltaY> ESC \
+ESC _ event=mouse ; id=<ID> ; kbmods=<KeyMods> ; coor=<X>,<Y> ; buttons=<ButtonState> ; iscroll=<DeltaX>,<DeltaY> ; fscroll=<DeltaX>,<DeltaY> ESC \
 ```
 
 Attribute                       | Description
 --------------------------------|------------
-`id=0`                          | Device group id.
-`kbmods=<KeyMods>`              | Keyboard modifiers (see Keyboard event).
-`coor=<X>,<Y>`                  | Pixel-wise coordinates of the mouse pointer. Each coordinate is represented in the form of a floating point value of the sum of the integer coordinate of the cell in the terminal window grid and the relative offset within the cell in the range `[0.0f, 1.0f)`.
-`buttons=<ButtonState>`         | Mouse button state.
-`scroll=<DeltaX>,<DeltaY>`      | Integer values of low resolution horizontal and vertical scroll deltas in integer 1/1 units (one scroll line corresponds to a value of 1).
-`finescroll=<DeltaX>,<DeltaY>`  | Integer values of high resolution horizontal and vertical scroll deltas in integer 1/120 units (one scroll line corresponds to a value of 120).
+`id=<ID>`                       | Device group id (unsigned integer value).
+`kbmods=<KeyMods>`              | Keyboard modifiers bit field (see Keyboard event).
+`coor=<X>,<Y>`                  | Pixel-wise 32-bit floating point coordinates of the mouse pointer relative to the console's text cell grid. The integer part corresponds to the cell coordinates, and the fractional part corresponds to the normalized position within the cell. The pointer's screen pixel coordinates can be calculated by multiplying these floating point values by the cell size. Receiving a NaN value is a signal that the mouse has left the window or disconnected.
+`buttons=<ButtonState>`         | Mouse buttons bit field.
+`iscroll=<DeltaX>,<DeltaY>`     | Low-resolution integer horizontal and vertical scroll deltas (one scroll line corresponds to a value of 1). Low-resolution scroll deltas increase as the values of high-resolution deltas accumulate, and are zeroed when the scroll direction changes.
+`fscroll=<DeltaX>,<DeltaY>`     | High-resolution 32-bit floating-point horizontal and vertical scroll deltas (one scroll line corresponds to a value of 1.0f).
 
 The mouse tracking event fires on any mouse activity, as well as on keyboard modifier changes.
 
@@ -393,12 +391,12 @@ Note: Mouse tracking will continue outside the terminal window as long as the mo
 ### Focus
 
 ```
-ESC _ event=focus ; id=0 ; state=<FocusState> ESC \
+ESC _ event=focus ; id=<ID> ; state=<FocusState> ESC \
 ```
 
 Attribute            | Description
 ---------------------|------------
-`id=0`               | Device group id.
+`id=<ID>`            | Device group id (unsigned integer value).
 `state=<FocusState>` | Terminal window focus:<br>\<FocusState\>=1 - Focused.<br>\<FocusState\>=0 - Unfocused.
 
 In response to the activation of `focus` tracking, the application receives a vt-sequence containing current focus state.
@@ -420,12 +418,12 @@ In response to the activation of `format` tracking, the application receives a v
 ### Clipboard
 
 ```
-ESC _ event=clipboard ; id=0 ; format=<ClipFormat> ; security=<SecLevel> ; data=<Data> ESC \
+ESC _ event=clipboard ; id=<ID> ; format=<ClipFormat> ; security=<SecLevel> ; data=<Data> ESC \
 ```
 
 Attribute             | Description
 ----------------------|------------
-`id=0`                | Device group id.
+`id=<ID>`             | Device group id (unsigned integer value).
 `format=<ClipFormat>` | Clipboard data format.
 `security=<SecLevel>` | Security level.
 `data=<Data>`         | Base64 encoded data.

src/netxs/desktopio/ansivt.hpp

@@ -260,13 +260,13 @@ namespace netxs::ansi
     static const auto paste_begin = "\033[200~"sv; // Bracketed paste begin.
     static const auto paste_end   = "\033[201~"sv; // Bracketed paste end.
 
-    static constexpr auto apc_prefix_mouse            = "event=mouse;"sv;
-    static constexpr auto apc_prefix_mouse_id         = "id="sv;         // ui32
-    static constexpr auto apc_prefix_mouse_kbmods     = "kbmods="sv;     // ui32
-    static constexpr auto apc_prefix_mouse_coor       = "coor="sv;       // fp32,fp32
-    static constexpr auto apc_prefix_mouse_buttons    = "buttons="sv;    // ui32
-    static constexpr auto apc_prefix_mouse_scroll     = "scroll="sv;     // si32,si32
-    static constexpr auto apc_prefix_mouse_finescroll = "finescroll="sv; // si32,si32
+    static constexpr auto apc_prefix_mouse         = "event=mouse;"sv;
+    static constexpr auto apc_prefix_mouse_id      = "id="sv;      // ui32
+    static constexpr auto apc_prefix_mouse_kbmods  = "kbmods="sv;  // ui32
+    static constexpr auto apc_prefix_mouse_coor    = "coor="sv;    // fp32,fp32
+    static constexpr auto apc_prefix_mouse_buttons = "buttons="sv; // ui32
+    static constexpr auto apc_prefix_mouse_iscroll = "iscroll="sv; // si32,si32
+    static constexpr auto apc_prefix_mouse_fscroll = "fscroll="sv; // fp32,fp32
 
     template<class Base>
     class basevt
@@ -622,27 +622,31 @@ namespace netxs::ansi
         template<class T>
         auto& mouse_vtm(T const& gear, fp2d coor) // escx: Mouse tracking report (vt-input-mode).
         {
-            //  ESC _ event=mouse ; id=0 ; kbmods=<KeyMods> ; coor=<X>,<Y> ; buttons=<ButtonState> ; scroll=<DeltaX>,<DeltaY> ST
-            auto wheelfp = netxs::saturate_cast<si32>(gear.m_sys.wheelfp * 120);
+            //  ESC _ event=mouse ; id=0 ; kbmods=<KeyMods> ; coor=<X>,<Y> ; buttons=<ButtonState> ; iscroll=<DeltaX>,<DeltaY> ; fscroll=<DeltaX>,<DeltaY> ST
             //todo make it fp2d
-            auto v1 = gear.m_sys.wheelsi;
-            auto h1 = 0;
-            auto v2 = wheelfp;
-            auto h2 = 0;
             auto x = ui32{};
             auto y = ui32{};
             ::memcpy(&x, &coor.x, sizeof(x));
             ::memcpy(&y, &coor.y, sizeof(y));
+            auto iv = gear.m_sys.wheelsi;
+            auto ih = 0;
+            auto fh = ui32{};
+            auto fv = ui32{};
+            ::memcpy(&fv, &gear.m_sys.wheelfp, sizeof(fv));
             if (gear.m_sys.hzwheel)
             {
-                std::swap(h1, v1);
-                std::swap(h2, v2);
+                std::swap(ih, iv);
+                std::swap(fh, fv);
             }
             add("\033_event=mouse;id=", gear.id, ";kbmods=", gear.m_sys.ctlstat, ";coor=");
-            utf::_to_hex(netxs::letoh(x), 4 * 2, [&](char c){ add(c); });
+            utf::_to_hex(x, 4 * 2, [&](char c){ add(c); });
             add(",");
-            utf::_to_hex(netxs::letoh(y), 4 * 2, [&](char c){ add(c); });
-            add(";buttons=", gear.m_sys.buttons, ";scroll=", h1, ",", v1, ";finescroll=", h2, ",", v2, "\033\\");
+            utf::_to_hex(y, 4 * 2, [&](char c){ add(c); });
+            add(";buttons=", gear.m_sys.buttons, ";iscroll=", ih, ",", iv, ";fscroll=");
+            utf::_to_hex(fh, 4 * 2, [&](char c){ add(c); });
+            add(",");
+            utf::_to_hex(fv, 4 * 2, [&](char c){ add(c); });
+            add("\033\\");
             return *this;
         }
         template<class T>

src/netxs/desktopio/application.hpp

@@ -22,7 +22,7 @@ namespace netxs::app
 
 namespace netxs::app::shared
 {
-    static const auto version = "v2025.08.14";
+    static const auto version = "v2025.08.15";
     static const auto repository = "https://github.com/directvt/vtm";
     static const auto usr_config = "~/.config/vtm/settings.xml"s;
     static const auto sys_config = "/etc/vtm/settings.xml"s;

src/netxs/desktopio/system.hpp

@@ -5707,9 +5707,9 @@ namespace netxs::os
                                             m.buttons = b.value();
                                         }
                                     }
-                                    else if (frag.starts_with(ansi::apc_prefix_mouse_scroll))
+                                    else if (frag.starts_with(ansi::apc_prefix_mouse_iscroll))
                                     {
-                                        frag.remove_prefix(ansi::apc_prefix_mouse_scroll.size());
+                                        frag.remove_prefix(ansi::apc_prefix_mouse_iscroll.size());
                                         if (auto h = utf::to_int<si32>(frag); h && frag)
                                         {
                                             frag.pop_front(); // Pop ','
@@ -5721,17 +5721,19 @@ namespace netxs::os
                                             }
                                         }
                                     }
-                                    else if (frag.starts_with(ansi::apc_prefix_mouse_finescroll))
+                                    else if (frag.starts_with(ansi::apc_prefix_mouse_fscroll))
                                     {
-                                        frag.remove_prefix(ansi::apc_prefix_mouse_finescroll.size());
-                                        if (auto h = utf::to_int<si32>(frag); h && frag)
+                                        frag.remove_prefix(ansi::apc_prefix_mouse_fscroll.size());
+                                        if (auto h = utf::to_int<ui32, 16>(frag); h && frag)
                                         {
                                             frag.pop_front(); // Pop ','
-                                            if (auto v = utf::to_int<si32>(frag))
+                                            if (auto v = utf::to_int<ui32, 16>(frag))
                                             {
                                                 //todo make it fp2d
-                                                m.hzwheel = h.value() != 0;
-                                                m.wheelfp = (m.hzwheel ? h.value() : v.value()) / 120.0;
+                                                auto fh = *reinterpret_cast<fp32*>(&h.value());
+                                                auto fv = *reinterpret_cast<fp32*>(&v.value());
+                                                m.hzwheel = fh != 0;
+                                                m.wheelfp = m.hzwheel ? fh : fv;
                                             }
                                         }
                                     }