Desktop: Performance: Faster startup and smaller application size

[personalizedrefrigerator]

Summary

This pull request bundles the desktop app with esbuild. On Linux, this:

• Reduces startup time by roughly 46% (see test steps below).
• Reduces the .AppImage size from 219 MiB to 162 MiB.

This change was also observed to improve performance on Windows 11:

• Reduces the portable app startup time by roughly 44%.
• Reduces the size of the portable application's .exe file from roughly 319 MiB to about 239 MiB.

Note

This pull request selects ESBuild over Webpack for two reasons:

• I'm more familiar with the APIs for customizing imports and postprocessing files — js-draw uses esbuild.
• esbuild should be much faster than Webpack.

Notes

• Bundling the application means that most requires and imports are resolved at compile time. This allows moving many dependencies from "dependencies" to "devDependencies", significantly reducing the application size. However, this means that shim.requireDynamic 1) usually cannot require relative paths and 2) can only require packages present in "dependencies".
  • Changes were made to remove several shim.requireDynamic calls.

Testing

Testing performance: Linux

I'm using the following script to compare the performance after this change with the performance of the 3.4.1 release:

Script

#!/bin/bash
# Commit with bundling
TO_COMMIT=61b6fe29aa23c3468cd1c931060ea90ecedc8281
# v3.4.1
FROM_COMMIT=24df67472623c4124a5262ee1631c4f205183683

# A diff that adds support for a --testing-start-perf flag.
# When the flag is provided, the app is closed just after the
# MainScreen component mounts (which happens after the main start
# logic runs).
START_PERF_DIFF=$(cat <<"EOF"
diff --git a/packages/app-desktop/gui/MainScreen.tsx b/packages/app-desktop/gui/MainScreen.tsx
index 2358def52..a0dbb4224 100644
--- a/packages/app-desktop/gui/MainScreen.tsx
+++ b/packages/app-desktop/gui/MainScreen.tsx
@@ -47,6 +47,17 @@ import PluginService from '@joplin/lib/services/plugins/PluginService';
 
 const ipcRenderer = require('electron').ipcRenderer;
 
+let testingStartPerf: boolean|null = null;
+const closeIfTestingStartPerf = () => {
+	testingStartPerf ??= bridge().processArgv().includes('--testing-start-perf');
+
+	// When testing start performance, close the app after everything is finished loading.
+	if (testingStartPerf) {
+		// Wait until the next iteration of the event loop, then close
+		requestAnimationFrame(() => bridge().electronApp().quit());
+	}
+};
+
 interface Props {
 	plugins: PluginStates;
 	pluginHtmlContents: PluginHtmlContents;
@@ -361,6 +372,7 @@ class MainScreenComponent extends React.Component<Props, State> {
 
 	public componentDidMount() {
 		window.addEventListener('keydown', this.layoutModeListenerKeyDown);
+		closeIfTestingStartPerf();
 	}
 
 	public componentWillUnmount() {
diff --git a/packages/lib/utils/processStartFlags.ts b/packages/lib/utils/processStartFlags.ts
index 3b6830c6f..3dd68bd6a 100644
--- a/packages/lib/utils/processStartFlags.ts
+++ b/packages/lib/utils/processStartFlags.ts
@@ -79,6 +79,12 @@ const processStartFlags = async (argv: string[], setDefaults = true) => {
 			continue;
 		}
 
+		if (arg === '--testing-start-perf') {
+			// Closes the app after startup completes
+			argv.splice(0, 1);
+			continue;
+		}
+
 		if (arg === '--update-geolocation-disabled') {
 			Note.updateGeolocationEnabled_ = false;
 			argv.splice(0, 1);

EOF
)

TEMP_DIR=/tmp/joplin-perf-test
mkdir "$TEMP_DIR"

cd ./joplin/

# Not actually CI, but this allows building the OneNote converter.
export IS_CONTINUOUS_INTEGRATION=1

function do_build() {
	echo "Building..."
	
	STATUS="$(git status --porcelain)"
	if [ "$STATUS" != "" ] ; then
		echo "ERROR: Found unstaged files $STATUS. Exiting."
		exit 1
	fi

	# Do a full rebuild to help ensure consistency
	git clean -dXf
	git reset --hard HEAD

	echo "Applying diff..."
	echo "$START_PERF_DIFF" | git apply -
	
	yarn
	if [ "$?" != "0" ] ; then
		echo "ERROR: yarn install failed"
		exit 1
	fi

	cd packages/app-desktop/ && yarn dist
	if [ "$?" != 0 ] ; then
		echo "ERROR: yarn dist failed"
		exit 1
	fi

	cd ../../
	mv packages/app-desktop/dist/Joplin-*.AppImage "$TEMP_DIR/Joplin.AppImage"

	# Clean any local changes
	git reset --hard HEAD
}

# Building
if [ ! -f "$TEMP_DIR/Joplin.old.AppImage" ]; then
	git checkout "$FROM_COMMIT" && do_build
	mv "$TEMP_DIR/Joplin.AppImage" "$TEMP_DIR/Joplin.old.AppImage"
else
	echo "Already built: Joplin.old.AppImage"
fi

if [ ! -f "$TEMP_DIR/Joplin.AppImage" ]; then
	git checkout "$TO_COMMIT" && do_build
else
	echo "Already built: Joplin.AppImage"
fi

# Performance testing
cd $TEMP_DIR
mkdir ./profile
PROFILE_DIR="$TEMP_DIR/profile"

echo "Setting up the profile..."
./Joplin.AppImage --testing-start-perf --profile $PROFILE_DIR

function timeStartup() {
	command time -f "%e" \
		--append "--output=./results$1" -- \
		./Joplin$2.AppImage --testing-start-perf --profile $PROFILE_DIR
	if [ "$?" != 0 ] ; then
		echo "ERROR: Failed."
		exit 1
	fi
}

# Reset the result files
echo "" > ./results-old
echo "" > ./results-new

for i in $(seq 1 10) ; do
	echo "Testing new..."
	timeStartup -new
	echo "Testing old..."
	timeStartup -old .old
done

echo "OLD"
cat ./results-old
echo "NEW"
cat ./results-new

At a high level, the script:

1. Builds a copy of the app from 24df674.
2. Builds a copy of the app from 61b6fe2.
3. Launches the app built in step 1 and times how long it takes for the main component to mount.
4. Launches the app built in step 2 and times how long it takes for the main component to mount.
5. Repeats steps 3-4 nine more times.

Results

Results on Fedora 42 — OS "Power Mode" set to "Performance"

Before: Startup time in seconds for Joplin v3.4.1:

5.25
5.10
5.00
5.10
4.97
5.08
5.14
5.22
5.40
5.10
Average: 5.14

After: Startup time in seconds for Joplin built from this pull request:

2.90
2.76
2.67
2.62
2.72
2.64
2.71
2.84
2.99
2.96
Average: 2.78

Percent change: $$\frac{\text{final}-\text{initial}}{\text{initial}} \times 100% = \frac{2.78-5.14}{5.14} \times 100%$$, which is a 45.9% decrease.

Results on Fedora 42 — OS "Power Mode" set to "Power Saver"

Before: Startup time in seconds for Joplin v3.4.1:

13.95
13.93
14.08
16.35
14.09
16.79
14.92
15.86
16.07
15.27
Average: 15.13

After: Startup time in seconds for Joplin built from this pull request:

8.03
7.85
7.55
7.89
7.95
7.26
7.24
10.02
7.75
7.83
Average: 7.94

Percent change: $$\frac{\text{final}-\text{initial}}{\text{initial}} \times 100% = \frac{7.94-15.13}{15.13} \times 100%$$, which is a 47.5% decrease.

On average, the startup time decreased by 46.7%.

Application size change: Comparing the before and after builds created by the above script,

• Before (Joplin.old.AppImage): 219 MiB
• After (Joplin.AppImage): 162 MiB

Testing performance: Windows

Similar tests were done on Windows. Tests were done using this script, run in git-bash.exe.

Results (Joplin portable on Windows 11): Roughly a 44% decrease in startup time

Before: Startup time in seconds for Joplin built from c9eb9af:

10.48
12.39
10.54
9.29
9.25
10.69
12.33
10.14
9.05
10.71
Average: 10.49

After: Startup time in seconds for Joplin built from this pull request:

5.64
6.74
6.42
5.54
4.73
5.89
6.89
6.26
5.26
5.58
Average: 5.90

Percent change: $$\frac{\text{final}-\text{initial}}{\text{initial}} \times 100% = \frac{5.90-10.49}{10.49} \times 100%$$, which is roughly a 43.8% decrease.

Size change: Comparing versions of Joplin Portable built by the above script:

• Before (JoplinPortable.exe): 319 MiB
• After (JoplinPortable.exe): 239 MiB

Regression testing

At present, this change mostly relies on automated tests. However, it has been manually verified that:

• On Fedora 42 (with an earlier commit) using the built .AppImage,
  • It is still possible to import a OneNote notebook (using one of the test OneNote notebooks in the tests/support subdirectory of the CLI app).
  • It is still possible to export a note to a single HTML file.
• On Windows 11 using the installed .exe:
  • It's possible to import app-cli/tests/enex_to_md/images_with_and_without_size.enex using file > import > "ENEX - Evernote Export File (as Markdown)".
  • It's possible to open the imported note in an external editor.
  • Making a change from the external editor (VSCode) and saving updates the original note.
  • Double-clicking the note opens it in a new window.

Planned manual testing to check for regressions in modified code:

• Check exporting from the CLI app:
  • JEX
  • HTML (not supported in the CLI app).
• Check JEX export from the Android app.

[personalizedrefrigerator]

Since both the main and renderer scripts are bundled, require('@electron/remote').require('./bridge') no longer results in the same instance of bridge as is used by the main process. The bridge must instead be transferred in a different way (see /services/bridge.ts).

[personalizedrefrigerator]

This particular change:

• Handles the case where @joplin/onenote-converter is not built by default in dev mode.
• Also may be required by the converter's WASM usage.

Since this isn't obvious, this should be specified in a .md file in readme/dev/spec.

[personalizedrefrigerator]

Done in 148e76f.

[personalizedrefrigerator]

CI is failing due to a scheduled brownout of the Windows 2019 action runner. This should be fixed by #12378.