Merge pull request #790 from o-sdn-o/gui-bridge

Fix mouse pointer position

Author: o-sdn-o

doc/vt-input-mode.md

@@ -14,24 +14,23 @@ Anyone who wants to:
 - Share applications on LAN (using inetd, netcat, etc).
 - Track every key press and key release.
 - Track position dependent keys such as WASD.
-- Distinguish between Left and Right physical keys.
+- Distinguish between physical Left and Right keyboard keys.
 - Get consistent output regardless of terminal window resize.
 - Track mouse on a pixel-wise level.
 - Track mouse outside the terminal window (getting negative coordinates).
-- Take advantage of high-resolution wheel scrolling.
+- Take advantage of high-resolution (fine) scrolling.
 - Track scrollback text manipulation.
 - Track application closing and system shutdown.
-- Be independent of operating system and third party libraries.
 
 Existing approaches have the following drawbacks:
 - There is no uniform way to receive keyboard events.
 - Window size tracking requires platform-specific calls with no way to synchronize the output.
-- Mouse tracking modes lack support for negative coordinates, and high-resolution wheel scrolling.
+- Mouse tracking modes lack support for negative coordinates, and high-resolution scrolling.
 - Bracketed paste mode does not support the transfer of binary data and data containing sequences of bracketed paste mode itself.
 
 ## Conventions
 
-- We use HEX-form of the uint32 for the 32-bit floating point value representation (IEEE-754 32-bit binary float, Little-Endian).
+- We use HEX-form of the uint32 for the 32-bit floating point value representation (IEEE-754 32-bit binary float, Little-Endian). For example, the floating point value `3.1415f` is represented as the unsigned integer in hex `40490E56` (decimal 1078529622).
 - Space characters are not used in sequence payloads and are only used for readability of the description.
 - //todo: keyboard only: All unescaped symbols outside of this protocol should be treated as clipboard pasted data.
 
@@ -121,7 +120,7 @@ ESC _ event=keyboard ; id=<ID> ; kbmods=<KeyMods> ; keyid=<KeyId> ; pressed=<Key
 
 Attribute                     | Description
 ------------------------------|------------
-`id=<ID>`                     | Device group id (unsigned integer value).
+`id=<ID>`                     | Device group id (unsigned integer).
 `kbmods=<KeyMods>`            | Keyboard modifiers bit field.
 `keyid=<KeyId>`               | Physical key ID.
 `pressed=<KeyDown>`           | Key state:<br>\<KeyDown\>=1 - Pressed.<br>\<KeyDown\>=0 - Released.
@@ -365,12 +364,12 @@ ESC _ event=mouse ; id=<ID> ; kbmods=<KeyMods> ; coor=<X>,<Y> ; buttons=<ButtonS
 
 Attribute                       | Description
 --------------------------------|------------
-`id=<ID>`                       | Device group id (unsigned integer value).
-`kbmods=<KeyMods>`              | Keyboard modifiers bit field (see Keyboard event).
-`coor=<X>,<Y>`                  | Pixel-wise 32-bit floating point coordinates of the mouse pointer relative to the console's text cell grid. The integer part corresponds to the cell coordinates, and the fractional part corresponds to the normalized position within the cell. The pointer's screen pixel coordinates can be calculated by multiplying these floating point values by the cell size. Receiving a NaN value is a signal that the mouse has left the window or disconnected.
-`buttons=<ButtonState>`         | Mouse buttons bit field.
-`iscroll=<DeltaX>,<DeltaY>`     | Low-resolution integer horizontal and vertical scroll deltas (one scroll line corresponds to a value of 1). Low-resolution scroll deltas increase as the values of high-resolution deltas accumulate, and are zeroed when the scroll direction changes.
-`fscroll=<DeltaX>,<DeltaY>`     | High-resolution 32-bit floating-point horizontal and vertical scroll deltas (one scroll line corresponds to a value of 1.0f).
+`id=<ID>`                       | Device group id (unsigned integer).
+`kbmods=<KeyMods>`              | Keyboard modifiers bit field (unsigned integer, the same value as in Keyboard event).
+`coor=<X>,<Y>`                  | 32-bit floating point coordinates of the mouse pointer relative to the console's text cell grid. The integer part corresponds to the cell coordinates, and the fractional part corresponds to the normalized position within the cell. The pointer's screen pixel coordinates can be calculated by multiplying these floating point values by the cell size. Receiving a NaN value is a signal that the mouse has left the window or disconnected.
+`buttons=<ButtonState>`         | Mouse buttons bit field (unsigned integer).
+`iscroll=<DeltaX>,<DeltaY>`     | Horizontal and vertical low-resolution scroll deltas in form of signed integers (one scroll line corresponds to a value of 1). Low-resolution scroll deltas increase as the values of high-resolution deltas accumulate, and are zeroed when the scroll direction changes.
+`fscroll=<DeltaX>,<DeltaY>`     | Horizontal and vertical high-resolution scroll deltas in form of 32-bit floating-point values (one scroll line corresponds to a value of 1.0f).
 
 The mouse tracking event fires on any mouse activity, as well as on keyboard modifier changes.
 
@@ -396,7 +395,7 @@ ESC _ event=focus ; id=<ID> ; state=<FocusState> ESC \
 
 Attribute            | Description
 ---------------------|------------
-`id=<ID>`            | Device group id (unsigned integer value).
+`id=<ID>`            | Device group id (unsigned integer).
 `state=<FocusState>` | Terminal window focus:<br>\<FocusState\>=1 - Focused.<br>\<FocusState\>=0 - Unfocused.
 
 In response to the activation of `focus` tracking, the application receives a vt-sequence containing current focus state.
@@ -423,7 +422,7 @@ ESC _ event=clipboard ; id=<ID> ; format=<ClipFormat> ; security=<SecLevel> ; da
 
 Attribute             | Description
 ----------------------|------------
-`id=<ID>`             | Device group id (unsigned integer value).
+`id=<ID>`             | Device group id (unsigned integer).
 `format=<ClipFormat>` | Clipboard data format.
 `security=<SecLevel>` | Security level.
 `data=<Data>`         | Base64 encoded data.

src/netxs/desktopio/ansivt.hpp

@@ -626,8 +626,16 @@ namespace netxs::ansi
             //todo make it fp2d
             auto x = ui32{};
             auto y = ui32{};
-            ::memcpy(&x, &coor.x, sizeof(x));
-            ::memcpy(&y, &coor.y, sizeof(y));
+            if (gear.mouse_disabled)
+            {
+                ::memcpy(&x, &fp32nan, sizeof(x));
+                ::memcpy(&y, &fp32nan, sizeof(y));
+            }
+            else
+            {
+                ::memcpy(&x, &coor.x, sizeof(x));
+                ::memcpy(&y, &coor.y, sizeof(y));
+            }
             auto iv = gear.m_sys.wheelsi;
             auto ih = 0;
             auto fh = ui32{};

src/netxs/desktopio/application.hpp

@@ -22,7 +22,7 @@ namespace netxs::app
 
 namespace netxs::app::shared
 {
-    static const auto version = "v2025.08.15";
+    static const auto version = "v2025.08.17";
     static const auto repository = "https://github.com/directvt/vtm";
     static const auto usr_config = "~/.config/vtm/settings.xml"s;
     static const auto sys_config = "/etc/vtm/settings.xml"s;

src/netxs/desktopio/console.hpp

@@ -540,7 +540,7 @@ namespace netxs::ui
                 if (ext_gear_id)
                 {
                     auto& gear = *gear_ptr;
-                    if (gear.m_sys.timecod != time{}) // Don't send mouse events if the mouse has not been used yet.
+                    if (gear.m_sys.timecod != time{} || gear.mouse_disabled) // Don't send mouse events if the mouse has not been used yet or disabled.
                     {
                         gear.fire_fast();
                         gear.fire(event_id);
@@ -569,15 +569,15 @@ namespace netxs::ui
             static const auto busy = cell{}.bgc(reddk).fgc(0xFFffffff);
             auto brush = gear.m_sys.buttons ? cell{ busy }.txt(64 + (char)gear.m_sys.buttons/*A-Z...*/)
                                             : idle;
-            auto area = rect{ gear.coord, dot_11 };
+            auto area = rect{ gear.coord + gear.owner.coor(), dot_11 };
             parent_canvas.fill(area, cell::shaders::fuse(brush));
         }
         void draw_mouse_pointer(face& parent_canvas)
         {
             for (auto& [ext_gear_id, gear_ptr] : gears)
             {
                 auto& gear = *gear_ptr;
-                if (gear.mouse_disabled) continue;
+                if (gear.mouse_disabled || std::isnan(gear.coord.x)) continue;
                 fill_pointer(gear, parent_canvas);
             }
         }
@@ -589,7 +589,7 @@ namespace netxs::ui
                 gear.board::shown = !gear.mouse_disabled &&
                                     (props.clip_preview_time == span::zero() ||
                                      props.clip_preview_time > stamp - gear.delta.stamp());
-                if (gear.board::shown)
+                if (gear.board::shown && !std::isnan(gear.coord.x))
                 {
                     auto coor = twod{ gear.coord } + dot_21 * 2;
                     auto full = gear.board::image.full();
@@ -609,7 +609,7 @@ namespace netxs::ui
                 auto& gear = *gear_ptr;
                 if (gear.mouse_disabled) continue;
                 auto [tooltip_page_sptr, tooltip_offset] = gear.tooltip.get_render_sptr_and_offset(props.tooltip_colors);
-                if (tooltip_page_sptr)
+                if (tooltip_page_sptr && !std::isnan(gear.coord.x))
                 {
                     auto& tooltip_page = *tooltip_page_sptr;
                     auto fs_area = full;
@@ -689,10 +689,6 @@ namespace netxs::ui
                     {
                         debug.output(canvas);
                     }
-                    if (props.legacy_mode & ui::console::mouse) // Render our mouse pointer.
-                    {
-                        draw_mouse_pointer(canvas);
-                    }
                     if (props.show_regions)
                     {
                         canvas.each([](cell& c)
@@ -705,6 +701,10 @@ namespace netxs::ui
                         });
                     }
                 }
+                if (props.legacy_mode & ui::console::mouse) // Render our mouse pointer.
+                {
+                    draw_mouse_pointer(canvas);
+                }
             }
             else
             {

src/netxs/desktopio/consrv.hpp

@@ -966,10 +966,10 @@ struct impl : consrv
         }
         void mouse(input::hids& gear, fp2d coord)
         {
-            if (std::isnan(coord.x)) // Forward a mouse halt event.
+            if (gear.mouse_disabled || std::isnan(coord.x)) // Forward a mouse halt event.
             {
                 auto lock = std::lock_guard{ locker };
-                auto r2 = nt::console::fp2d_mouse_input{ .coord = coord };
+                auto r2 = nt::console::fp2d_mouse_input{ .coord = { fp32nan, fp32nan} };
                 stream.emplace_back(*reinterpret_cast<INPUT_RECORD*>(&r2));
                 ondata.reset();
                 signal.notify_one();

src/netxs/desktopio/system.hpp

@@ -5740,7 +5740,16 @@ namespace netxs::os
                                 });
                                 m.changed++;
                                 m.timecod = datetime::now();
-                                mouse(m);
+                                if (std::isnan(m.coordxy.x))
+                                {
+                                    m.enabled = input::hids::stat::halt; // Send a mouse halt event.
+                                    mouse(m);
+                                    m.enabled = input::hids::stat::ok;
+                                }
+                                else
+                                {
+                                    mouse(m);
+                                }
                             }
                             else if (t == type::mouse) // ESC [ < ctrl ; xpos ; ypos M
                             {

src/netxs/desktopio/terminal.hpp

@@ -517,6 +517,17 @@ namespace netxs::ui
                 state = (mode)(state & ~(m));
                 if (!state) token.clear();
                 owner.selection_selmod(smode);
+                auto gates = owner.base::riseup(tier::request, e2::form::state::keybd::enlist); // Take all foci.
+                for (auto gate_id : gates) // Reset double click state for all gears.
+                {
+                    if (auto gear_ptr = owner.base::getref<hids>(gate_id))
+                    {
+                        for (auto& [bttn_id, s] : gear_ptr->stamp) // Reset double click state. The issue is related to Far Manager, which changes the mouse tracking mode before releasing the button when double-clicking.
+                        {
+                            s.count = !!s.count; // Set to 1 if non zero.
+                        }
+                    }
+                }
             }
             void setmode(prot p) { encod = p; }
         };

src/vtm.hpp

@@ -1669,12 +1669,12 @@ namespace netxs::app::vtm
                             for (auto& [ext_gear_id, gear_ptr] : usergate.gears)
                             {
                                 auto& gear = *gear_ptr;
-                                if (gear.mouse_disabled) continue;
+                                if (gear.mouse_disabled || std::isnan(gear.coord.x)) continue;
                                 auto coor = twod{ gear.coord } + gear.owner.coor();
                                 coor.y -= 1;
                                 coor.x -= half_x;
                                 user_name.move(coor);
-                                parent_canvas.fill(user_name, cell::shaders::contrast);
+                                parent_canvas.fill(user_name, cell::shaders::contrast); //todo revise: segfault?
                                 usergate.fill_pointer(gear, parent_canvas);
                             }
                         }