Improve Multi-Line Braille Support

[emassey0135]

Is your feature request related to a problem? Please describe.

I am writing a BRLAPI server that runs on the Monarch, and I can connect to it using BRLTTY by setting the driver to ba and the ba:host parameter to the IP address of the Monarch. However, when I run BRLTTY like this and connect to it with NVDA, NVDA doesn't make good use of the 10 lines of 32 cells on the Monarch. For example, when I type two lines of text into Notepad, I can only read one line at a time on the display, and in other Windows it only displays one control at a time, even though it may use two or three lines to display very long controls. I know this is an issue with NVDA, not my server or BRLTTY, because when I run BRLTTY without NVDA connected to it, it uses the entire display area to display the terminal, on both Windows and Linux.

Describe the solution you'd like

When focused on a multi-line text field, NVDA should display as many lines as it can if connected to a multi-line Braille display, and panning should either move the display up and down by one line, or display the previous or next page of Braille (10 line area or however many lines the display has), and cursor routing should be able to move the cursor to a different line. Also, tables should be displayed with each row on a separate Braille line (or a few if there's not enough space) with each column starting at the same Braille cell between lines, just like in hardcopy Braille books and documents. In addition, NVDA could display other controls in the window above and below the Braille line/lines containing the current control, similarly to how the Monarch displays its own menu and Android applications, with the current control having dots 7 and 8 set under it or another focus indicator. Routing the cursor to another control would move the focus there. This would also be good for showing other items in list views, list boxes, and combo boxes. Perhaps NVDA could also allow you to split the display between windows, for example giving the top 5 lines or leftmost 16 columns to Microsoft Word and the bottom 5 lines or rightmost 16 columns to Outlook, so you can be editing a Word document in one area while you read information from an email message from the other. It doesn't have to be half and half either, and routing the cursor to somewhere in the window not currently focused could change the currently focused window to that one. Perhaps the object model could also be represented; for example, the top line of the display could say toolbar, then below it you could have a document. When you perform a command to expand the toolbar, it would change into three lines of Braille displaying some buttons plus an edit field. If you expand the edit field, you could read the text it contains. This would allow a user to jump between different areas of a window a lot faster, and be able to understand the layout of a window without navigating around everything first. This would be especially useful if controls didn't always take a whole line, and perhaps also could be shortened or abbriviated before the user expands them. For example, perhaps buttons with icons could be shown as Braille symbols representing those icons, and when they are expanded the text label is shown, so that a toolbar with a lot of buttons could be shown as one or two lines of Braille with symbols the user will have learned to recognize over time, instead of many lines of text labels.

Describe alternatives you've considered

I am not sure there are any, except just using the terminal with editors like Emacs.

Additional context

JAWS will soon be adding support for multi-line Braille displays including the Monarch, including support for displaying tables as I described above. Also, these features would also be beneficial for people who use a Dot Pad, Orbit Slate, Cadence, or any other present or future multi-line display.

[josephsl]

Hi,

Which version of NVDA did you test with - 2025.1.x, 2024.4.x, 2025.2 beta 1, or 2025.3 alpha? This matters - multi-line display support was added in 2025.1 (if I read the what's new document correctly). Also, does Monarch expose its own API/protocol besides the one used by BRLAPI?

Thanks.

[emassey0135]

I tested this with NVDA 2025.1.2. Currently, the Monarch does not have a Braille terminal application yet, and so does not have any protocol that devices can use to display Braille on it. HumanWare must be working on one for the JAWS support of the Monarch that is being worked on, but this is not in the latest version of the software yet, which is why I'm writing my own Braille terminal app for it using BRLAPI.

[SaschaCowley]

Hi @emassey0135,

Thanks for sharing these ideas with us! We are excited about the potential of multi-line braille support, and are working on bringing this to NVDA. You've outlined some really interesting usage patterns here, and we look forward to working with you to prototype them and potentially add them to NVDA.

We have converted the issue to a discussion because it is currently very nebulous - this is not something we can resolve with a single pull request. We really appreciate the thought you have put into the initial issue, and look forward to continued discussion!

Thanks,
NV Access software developers