Performance regression in insiders

[bpasero]

Overall ellapsed trend:

Evident increase in time to load workbench bundle:

[bpasero]

Newer runs are quite better at least on Windows, still on the edge of being slow (1.9s):

[jrieken]

Looks like code loading times went (consistently) up with b60db78. The size of workbench-main didn't really change - it's 10.3MB for both versions. So, this is either slower code loading by electron (cc @deepak1556) or more code execution while loading

[jrieken]

change between those two builds: 502ac5b...b60db78

[jrieken]

I can reproduce locally when running the builds in comparison. There is a slowdown on first run (which doesn't have cached data) and on subsequent runs. The slowdown seems constant which indicates towards a code execution problem

502ac	b60db
366	386
136	169
140	158
135	156

[bpasero]

I also did some testing today. First of all, even on the macOS perf machine, we lost time in the require of workbench main (timers.ellapsedrequire). The increase is quite stable and has not recovered:

I did compare startup profiles on macOS for the fast version on March 1st and then today but could not spot something obvious. My hope was that maybe the _invokeFactory call would be slower, hinting towards an expensive module, but that was not showing up. We did however add more contributions to the workbench, including interactive.

The findings around cached data are good, I did not specifically look into that. To my knowledge there was no Electron update, not even a minor one. But we should double check that for example no Profile-Guided-Optimization change sneaked into the insiders build.

[deepak1556]

Runtime has not changed since v1.76 branched off, we did have a small duration of Electron 22 being merged into main but then reverted to the same Electron 19 version as v1.76 branch.

But we should double check that for example no Profile-Guided-Optimization change sneaked into the insiders build.

No change in this area as well

[lorand-horvath]

@bpasero I'd like to add something that I've noticed in Insiders since around March 9-10, which might be related to the perf regression you've measured since then.
Not sure if you're familiar with this issue #130320 (comment) which basically pops up a notification every 5-20 minutes, saying that the terminal's host process is unresponsive - so I have to restart it to be able to use the terminal again (the time it takes for the notification to re-appear varies a lot and pops up mostly when VSCode is out of focus, so when I'm working in another app on the secondary monitor).

My point here is that around March 9 (maybe a couple of days before) the popup stopped appearing in Insiders - mostly - not appearing for several hours, even days on end. However, I think the error still occurs in the background, without a notification being shown.

As additional information, if you don't want to search the long thread, here's a typical example of a complete error notification appearing before March 9 in Insiders and also in the official VSCode 1.76 on a regular basis (ongoing since June 2022) #130320 (comment)

I'm not sure if this has anything to do with the performance regression, I thought I'd mention it in case it proves to be somehow related.
Also worth mentioning that this has been reported many times in other issues, but they are always closed as duplicate of #130320 - so it's not at all a rare occurrence - happens on Win 10, 7, Linux and I think I've seen a report on MacOS as well.