os.system('cls') does not clear the screen when redirect is enabled

[huguesv]

from customer feedback at https://developercommunity.visualstudio.com/content/problem/992505/python-clear-screen-broken.html

Environment data

• PTVSD version: debugpy 1.0.0b5
• OS and version: windows
• Python version (& distribution if applicable, e.g. Anaconda): 3.7
• Using VS Code or Visual Studio: Visual Studio

Actual behavior

os.system('clr') prints control character and does not clear the screen

Expected behavior

os.system('clr') clears the console window when debugging

Steps to reproduce:

1. Make sure redirect output is enabled

2. Print some stuff and clear the screen

Note that disabling redirect output solves the issue.

[fabioz]

Well, this happens because the approach to redirect output changed.

Previously, we redirected sys.stdout and sys.stderr internally... while this worked in general it did fail on some cases:

• C/C++ streams were not redirected
• if the user did fiddle with the sys module it could stop -- for instance, doing something as reload(sys).
• The output shown sometimes wasn't exactly what Python itself would give (in particular, when using threading the output wasn't exactly the same -- the contents were, but the timing could be different)

So, the new approach launches the subprocess and reads all the contents for the output redirect through an os.pipe (i.e.: https://github.com/microsoft/debugpy/blob/master/src/debugpy/launcher/debuggee.py#L45)... this does have downsides too (as is the case for os.system('cls') not working).

@karthiknadig @int19h I'm not sure the best way to deal with this. I think that the current approach is better for the majority of the cases, but it does seem like the old one was useful on some occasions too...

Maybe we should add a different flag for the user to choose which redirectOutput flavor he wants? (i.e.: redirect by pipe or redirect by sys streams) -- the old code is still supported to redirect in pydevd (and I think it needs to be the one used when using redirectOutput on attach anyways), so, adding that option should be feasible to do in the adapter.

[int19h]

Yeah, we're using pydevd redirection logic for "attach".

We can certainly do a flag for it - it's just that so far nobody has actually asked for one. In VSCode, it's just not particularly useful, because normally the app is running in the integrated terminal, which already captures the output. OTOH if it's not in the terminal, but writing to Debug Console, then there's no expectation for cls and the like to work. In VS, however, since there's no integrated terminal, there's more incentive for output capture.

The Node.js debugger has a similar arrangement with output capture, but instead of using a boolean yes/no flag like we do, they have a property named "outputCapture" that takes string values - "std" to capture at stdout/err level, and "console" to capture at Node API level. We could adopt this approach in lieu of "redirectOutput", and add another option for no capture at all; "redirectOutput" would then become a legacy partial alias.

[int19h]

BTW, another possibility is to try to handle this in the redirection code. If cls actually outputs an ANSI escape sequence there when it sees stdout redirected, we can detect that on the other end of the pipe, in the launcher. Since the launcher has the actual stdout, it can check whether that is a terminal or not, and translate it back to whatever cls normally does (or even just spawn it directly).

However, this only fixes clearing the screen. There are other things that can be done with a real console, that are effectively suppressed when output is redirected on stdio level - for example, random access to the screen buffer, common for TUI apps.

[huguesv]

Also reported at https://developercommunity.visualstudio.com/content/problem/1048897/issue-with-console-commands-python-3.html

[judej]

We need to see if this is still an issue on VS 2022.

[int19h]

As this is the interaction between debugpy and the debuggee running in an external console window, it's not VS-specific. So yes, it's still a problem.

I think the easiest fix would be to have the three-valued switch as described in the comment above.

In addition, given that VS 2019+ has an integrated terminal, we should see if using "console": "integratedTerminal" makes use of that. If it does, then we should probably make that the default, same as in VSCode.

[int19h]

VS 2022 still doesn't support "console": "integratedTerminal".