sea: use JSON configuration and blob content for SEA

Author: joyeecheung

doc/api/cli.md

@@ -588,6 +588,16 @@ added: v16.6.0
 
 Use this flag to disable top-level await in REPL.
 
+### `--experimental-sea-config`
+
+<!-- YAML
+added: REPLACEME
+-->
+
+Use this flag to generate a blob that can be injected in to the Node.js
+binary to produce a [single executable application][]. See the documentation
+about [this configuration][`--experimental-sea-config`] for details.
+
 ### `--experimental-shadow-realm`
 
 <!-- YAML
@@ -2550,6 +2560,7 @@ done
 [`--cpu-prof-dir`]: #--cpu-prof-dir
 [`--diagnostic-dir`]: #--diagnostic-dirdirectory
 [`--experimental-wasm-modules`]: #--experimental-wasm-modules
+[`--experimental-sea-config`]: single-executable-applications.md#generating-single-executable-preparation-blobs
 [`--heap-prof-dir`]: #--heap-prof-dir
 [`--import`]: #--importmodule
 [`--openssl-config`]: #--openssl-configfile
@@ -2587,6 +2598,7 @@ done
 [scavenge garbage collector]: https://v8.dev/blog/orinoco-parallel-scavenger
 [security warning]: #warning-binding-inspector-to-a-public-ipport-combination-is-insecure
 [semi-space]: https://www.memorymanagement.org/glossary/s.html#semi.space
+[single executable applications]: single-executable-applications.md
 [test reporters]: test.md#test-reporters
 [timezone IDs]: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
 [tracking issue for user-land snapshots]: https://github.com/nodejs/node/issues/44014

doc/api/single-executable-applications.md

@@ -10,15 +10,17 @@ This feature allows the distribution of a Node.js application conveniently to a
 system that does not have Node.js installed.
 
 Node.js supports the creation of [single executable applications][] by allowing
-the injection of a JavaScript file into the `node` binary. During start up, the
-program checks if anything has been injected. If the script is found, it
-executes its contents. Otherwise Node.js operates as it normally does.
+the injection of a blob prepared by Node.js, which can contain a bundled script,
+into the `node` binary. During start up, the program checks if anything has been
+injected. If the blob is found, it executes the script in the blob. Otherwise
+Node.js operates as it normally does.
 
-The single executable application feature only supports running a single
-embedded [CommonJS][] file.
+The single executable application feature currently only supports running a
+single embedded script using the [CommonJS][] module system.
 
-A bundled JavaScript file can be turned into a single executable application
-with any tool which can inject resources into the `node` binary.
+Users can create a single executable application from their bundled script
+with the `node` binary itself and any tool which can inject resources into the
+binary.
 
 Here are the steps for creating a single executable application using one such
 tool, [postject][]:
@@ -28,12 +30,23 @@ tool, [postject][]:
    $ echo 'console.log(`Hello, ${process.argv[2]}!`);' > hello.js
    ```
 
-2. Create a copy of the `node` executable and name it according to your needs:
+2. Create a configuration file building a blob that can be injected into the
+   single executable application:
+   ```console
+   $ echo '{ "main": "hello.js", "output": "sea-prep.blob" }' > sea-config.json
+   ```
+
+3. Generate the blob to be injected:
+   ```console
+   $ node --experimental-sea-config sea-config.json
+   ```
+
+4. Create a copy of the `node` executable and name it according to your needs:
    ```console
    $ cp $(command -v node) hello
    ```
 
-3. Remove the signature of the binary:
+5. Remove the signature of the binary:
 
    * On macOS:
 
@@ -50,35 +63,35 @@ tool, [postject][]:
    $ signtool remove /s hello
    ```
 
-4. Inject the JavaScript file into the copied binary by running `postject` with
+6. Inject the blob into the copied binary by running `postject` with
    the following options:
 
    * `hello` - The name of the copy of the `node` executable created in step 2.
-   * `NODE_JS_CODE` - The name of the resource / note / section in the binary
-     where the contents of the JavaScript file will be stored.
-   * `hello.js` - The name of the JavaScript file created in step 1.
-   * `--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2` - The
+   * `NODE_SEA_BLOB` - The name of the resource / note / section in the binary
+     where the contents of the blob will be stored.
+   * `hello.js` - The name of the blob created in step 1.
+   * `--sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2` - The
      [fuse][] used by the Node.js project to detect if a file has been injected.
-   * `--macho-segment-name NODE_JS` (only needed on macOS) - The name of the
-     segment in the binary where the contents of the JavaScript file will be
+   * `--macho-segment-name NODE_SEA` (only needed on macOS) - The name of the
+     segment in the binary where the contents of the blob will be
      stored.
 
    To summarize, here is the required command for each platform:
 
    * On systems other than macOS:
      ```console
-     $ npx postject hello NODE_JS_CODE hello.js \
-         --sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2
+     $ npx postject hello NODE_SEA_BLOB hello.js \
+         --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2
      ```
 
    * On macOS:
      ```console
-     $ npx postject hello NODE_JS_CODE hello.js \
-         --sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \
-         --macho-segment-name NODE_JS
+     $ npx postject hello NODE_SEA_BLOB hello.js \
+         --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \
+         --macho-segment-name NODE_SEA
      ```
 
-5. Sign the binary:
+7. Sign the binary:
 
    * On macOS:
 
@@ -95,12 +108,33 @@ tool, [postject][]:
    $ signtool sign /fd SHA256 hello
    ```
 
-6. Run the binary:
+8. Run the binary:
    ```console
    $ ./hello world
    Hello, world!
    ```
 
+## Generating single executable preparation blobs
+
+Single executable preparation blobs that are injected into the application can
+be generated using the `--experimental-sea-config` flag of the Node.js binary
+that will be used to build the single executable. It takes a path to a
+configuration file in JSON format. If the path passed to it isn't absolute,
+Node.js will use the path relative to the current working directory.
+
+The configuration currently reads the following top-level fields:
+
+```json
+{
+  "main": "/path/to/bundled/script.js",
+  "output": "/path/to/write/the/generated/blob.blob"
+}
+```
+
+If the paths are not absolute, Node.js will use the path relative to the
+current working directory. The version of the Node.js binary used to produce
+the blob must be the same as the one to which the blob will be injected.
+
 ## Notes
 
 ### `require(id)` in the injected module is not file based
@@ -135,15 +169,16 @@ of [`process.execPath`][].
 ### Single executable application creation process
 
 A tool aiming to create a single executable Node.js application must
-inject the contents of a JavaScript file into:
+inject the contents of the blob prepared with `--experimental-sea-config"`
+into:
 
-* a resource named `NODE_JS_CODE` if the `node` binary is a [PE][] file
-* a section named `NODE_JS_CODE` in the `NODE_JS` segment if the `node` binary
+* a resource named `NODE_SEA_BLOB` if the `node` binary is a [PE][] file
+* a section named `NODE_SEA_BLOB` in the `NODE_SEA` segment if the `node` binary
   is a [Mach-O][] file
-* a note named `NODE_JS_CODE` if the `node` binary is an [ELF][] file
+* a note named `NODE_SEA_BLOB` if the `node` binary is an [ELF][] file
 
 Search the binary for the
-`NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2:0` [fuse][] string and flip the
+`NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2:0` [fuse][] string and flip the
 last character to `1` to indicate that a resource has been injected.
 
 ### Platform support

doc/contributing/maintaining-single-executable-application-support.md

@@ -66,7 +66,7 @@ To disable single executable application support, build Node.js with the
 ## Implementation
 
 When built with single executable application support, the Node.js process uses
-[`postject-api.h`][] to check if the `NODE_JS_CODE` section exists in the
+[`postject-api.h`][] to check if the `NODE_SEA_BLOB` section exists in the
 binary. If it is found, it passes the buffer to
 [`single_executable_application.js`][], which executes the contents of the
 embedded script.

node.gyp

@@ -82,6 +82,8 @@
       'src/js_stream.cc',
       'src/json_utils.cc',
       'src/js_udp_wrap.cc',
+      'src/json_parser.h',
+      'src/json_parser.cc',
       'src/module_wrap.cc',
       'src/node.cc',
       'src/node_api.cc',

src/json_parser.cc

@@ -0,0 +1,80 @@
+#include "json_parser.h"
+#include "node_errors.h"
+#include "node_v8_platform-inl.h"
+#include "util-inl.h"
+
+namespace node {
+using v8::ArrayBuffer;
+using v8::Context;
+using v8::Isolate;
+using v8::Local;
+using v8::Object;
+using v8::String;
+using v8::Value;
+
+static Isolate* NewIsolate(v8::ArrayBuffer::Allocator* allocator) {
+  Isolate* isolate = Isolate::Allocate();
+  CHECK_NOT_NULL(isolate);
+  per_process::v8_platform.Platform()->RegisterIsolate(isolate,
+                                                       uv_default_loop());
+  Isolate::CreateParams params;
+  params.array_buffer_allocator = allocator;
+  Isolate::Initialize(isolate, params);
+  return isolate;
+}
+
+void JSONParser::FreeIsolate(Isolate* isolate) {
+  per_process::v8_platform.Platform()->UnregisterIsolate(isolate);
+  isolate->Dispose();
+}
+
+JSONParser::JSONParser()
+    : allocator_(ArrayBuffer::Allocator::NewDefaultAllocator()),
+      isolate_(NewIsolate(allocator_.get())),
+      handle_scope_(isolate_.get()),
+      context_(isolate_.get(), Context::New(isolate_.get())),
+      context_scope_(context_.Get(isolate_.get())) {}
+
+bool JSONParser::Parse(const std::string& content) {
+  DCHECK(!parsed_);
+
+  Isolate* isolate = isolate_.get();
+  Local<Context> context = context_.Get(isolate);
+
+  // It's not a real script, so don't print the source line.
+  errors::PrinterTryCatch bootstrapCatch(
+      isolate, errors::PrinterTryCatch::kDontPrintSourceLine);
+  Local<Value> json_string_value;
+  Local<Value> result_value;
+  if (!ToV8Value(context, content).ToLocal(&json_string_value) ||
+      !json_string_value->IsString() ||
+      !v8::JSON::Parse(context, json_string_value.As<String>())
+           .ToLocal(&result_value) ||
+      !result_value->IsObject()) {
+    return false;
+  }
+  content_.Reset(isolate, result_value.As<Object>());
+  parsed_ = true;
+  return true;
+}
+
+std::optional<std::string> JSONParser::GetTopLevelField(
+    const std::string& field) {
+  Isolate* isolate = isolate_.get();
+  Local<Context> context = context_.Get(isolate);
+  Local<Object> content_object = content_.Get(isolate);
+  Local<Value> value;
+  // It's not a real script, so don't print the source line.
+  errors::PrinterTryCatch bootstrapCatch(
+      isolate, errors::PrinterTryCatch::kDontPrintSourceLine);
+  if (!content_object
+           ->Get(context, OneByteString(isolate, field.c_str(), field.length()))
+           .ToLocal(&value) ||
+      !value->IsString()) {
+    return {};
+  }
+  Utf8Value utf8_value(isolate, value);
+  return utf8_value.ToString();
+}
+
+}  // namespace node

src/json_parser.h

@@ -0,0 +1,39 @@
+#ifndef SRC_JSON_PARSER_H_
+#define SRC_JSON_PARSER_H_
+
+#if defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
+
+#include <memory>
+#include <optional>
+#include <string>
+#include "util.h"
+#include "v8.h"
+
+namespace node {
+// This is intended to be used to get some top-level fields out of a JSON
+// without having to spin up a full Node.js environment that unnecessarily
+// complicates things.
+class JSONParser {
+ public:
+  JSONParser();
+  ~JSONParser() {}
+  bool Parse(const std::string& content);
+  std::optional<std::string> GetTopLevelField(const std::string& field);
+
+ private:
+  // We might want a lighter-weight JSON parser for this use case. But for now
+  // using V8 is good enough.
+  static void FreeIsolate(v8::Isolate* isolate);
+  std::unique_ptr<v8::ArrayBuffer::Allocator> allocator_;
+  DeleteFnPtr<v8::Isolate, FreeIsolate> isolate_;
+  v8::HandleScope handle_scope_;
+  v8::Global<v8::Context> context_;
+  v8::Context::Scope context_scope_;
+  v8::Global<v8::Object> content_;
+  bool parsed_ = false;
+};
+}  // namespace node
+
+#endif  // defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
+
+#endif  // SRC_JSON_PARSER_H_

src/node.cc

@@ -1239,6 +1239,11 @@ static ExitCode StartInternal(int argc, char** argv) {
 
   uv_loop_configure(uv_default_loop(), UV_METRICS_IDLE_TIME);
 
+  std::string sea_config = per_process::cli_options->experimental_sea_config;
+  if (!sea_config.empty()) {
+    return sea::BuildSingleExecutableBlob(sea_config);
+  }
+
   // --build-snapshot indicates that we are in snapshot building mode.
   if (per_process::cli_options->per_isolate->build_snapshot) {
     if (result->args().size() < 2) {

src/node_errors.cc

@@ -246,14 +246,19 @@ void PrintStackTrace(Isolate* isolate, Local<StackTrace> stack) {
 std::string FormatCaughtException(Isolate* isolate,
                                   Local<Context> context,
                                   Local<Value> err,
-                                  Local<Message> message) {
+                                  Local<Message> message,
+                                  bool add_source_line = true) {
+  std::string result;
   node::Utf8Value reason(isolate,
                          err->ToDetailString(context)
                              .FromMaybe(Local<String>()));
-  bool added_exception_line = false;
-  std::string source =
-      GetErrorSource(isolate, context, message, &added_exception_line);
-  std::string result = source + '\n' + reason.ToString() + '\n';
+  if (add_source_line) {
+    bool added_exception_line = false;
+    std::string source =
+        GetErrorSource(isolate, context, message, &added_exception_line);
+    result = source + '\n';
+  }
+  result += reason.ToString() + '\n';
 
   Local<v8::StackTrace> stack = message->GetStackTrace();
   if (!stack.IsEmpty()) result += FormatStackTrace(isolate, stack);
@@ -1209,6 +1214,19 @@ void TriggerUncaughtException(Isolate* isolate, const v8::TryCatch& try_catch) {
                            false /* from_promise */);
 }
 
+PrinterTryCatch::~PrinterTryCatch() {
+  if (!HasCaught()) {
+    return;
+  }
+  std::string str =
+      FormatCaughtException(isolate_,
+                            isolate_->GetCurrentContext(),
+                            Exception(),
+                            Message(),
+                            print_source_line_ == kPrintSourceLine);
+  PrintToStderrAndFlush(str);
+}
+
 }  // namespace errors
 
 }  // namespace node