Excessive file system IO

[octref]

#784 (comment)

VueServer: 181,000 IO reads, ~10% CPU
TSServer: 1800 reads, ~1% CPU

The root cause seems to be that Vetur loads all js/ts files from workspace, and this will result in TS Server loading all their dependencies:

vetur/server/src/services/typescriptService/serviceHost.ts

Line 69 in 7fa2ead

const files = parsedConfig.fileNames;

[octref]

#1278, VSIX to test: https://github.com/vuejs/vetur/releases/tag/perf

Only one failing test so far, which I believe can be fixed by always including vuex and vue-router's definitions files in the scriptFiles.

[octref]

Hmm, all responses for getScriptSnapshot should be cached as well.

[octref]

Cached all access to node_modules. In macOS Instruments, I see 10x reduced file system access (basically all reads in node_modules are cached).

I updated the pre-release VSIX. @NoahStahl, can you give it a try?

[petternordholm]

@octref CPU usage is really low with the new version. Using the latest version from scriptFiles branch (latest commit is "Cache file system acccess).

[octref]

Sounds good. Will do a release tomorrow.

[petternordholm]

In our patched version, we are at the moment using a slightly different approach to minimize IO and speed up completion.

petternordholm@60eef39

The commit above enables us to use a vetur specific tsconfig.json file and in this file, we only include the bare minimum needed for code completion from vue files.

{
  "extends": "./tsconfig",
  "include": [
    "**/*service.ts",
    "**/*utils.ts",
    "**/*helper.ts"
  ]
}

The vetur specific tsconfig enables us to keep existing completion ts files.

All our business logic is accessed via "services" from our vue files. So, the vetur specific configuration file also forces the developer to follow our development guidelines.

[NoahStahl]

Thanks for the progress @octref. I tried the updated version but it doesn't work at least on Windows, in particular validation is now always skipped. Looks like it is listing files using fsPath, but then using path to compare against. In my environment (Win 10), fsPath and path appear to return different values, so filePaths.includes(filePath) is always false due to slashes. Can it be updated to use normalized paths?

Relevant sections:

vetur/server/src/modes/script/javascript.ts

Line 604 in 3600c56

export function languageServiceIncludesFile(ls: ts.LanguageService, documentUri: string): boolean {

vetur/server/src/utils/paths.ts

Line 43 in 3600c56

export function getFileFsPath(documentUri: string): string {

[NoahStahl]

After dropping in the slash normalization that typescript uses, perhaps a bit hacky:

605: const filePaths = ls.getProgram()!.getRootFileNames().map(p => p.replace(/\\/g, '/'));

It works. And performance is much improved. Roughly, it looks like about 80% less IO, 40-50% less CPU utilization, and significantly less memory use. Most importantly, error checking and completion is usable again with responses coming back quickly. Awesome!

I still suspect there's more room to optimize, as the usage is still multiples of what .ts files do for same input. But, this change alone will make a huge difference for me. Thanks!

[octref]

That function should have used getFileFsPath:

export function languageServiceIncludesFile(ls: ts.LanguageService, documentUri: string): boolean {
  const filePaths = ls.getProgram()!.getRootFileNames();
  const filePath = getFileFsPath(documentUri);
  return filePaths.includes(filePath);
}

filePaths have the fsPath type of paths, right?

vetur/server/src/utils/paths.ts

Lines 34 to 40 in 3600c56

	* - Windows
	* ```
	* > Uri.parse('file:///c%3A/foo/bar.vue').fsPath
	* 'c:\\foo\\bar.vue' (\\ escapes to \)
	* > Uri.parse('file:///c%3A/foo/bar.vue').path
	* '/c:/foo/bar.vue'
	* ```

Let me setup #1266 and fix it tomorrow.