Skip to content

Commit 5335c10

Browse files
committed
fs: port SonicBoom module to fs module as Utf8Stream
As a first step to porting portions of the pino structured logger into the runtime, this commit ports the SonicBoom module to the fs module as Utf8Stream. This is a faithful port of the SonicBoom module with some modern updates, such as converting to a Class and using Symbol.dispose. The bulk of the implementation is unchanged from the original. PR-URL: nodejs#58897 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Vinícius Lourenço Claro Cardoso <[email protected]>
1 parent c8b1fbe commit 5335c10

18 files changed

+2888
-0
lines changed

doc/api/fs.md

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7710,6 +7710,186 @@ added:
77107710
77117711
Type of file system.
77127712
7713+
### Class: `fs.Utf8Stream`
7714+
7715+
<!-- YAML
7716+
added: REPLACEME
7717+
-->
7718+
7719+
> Stability: 1 - Experimental
7720+
7721+
An optimized UTF-8 stream writer that allows for flushing all the internal
7722+
buffering on demand. It handles `EAGAIN` errors correctly, allowing for
7723+
customization, for example, by dropping content if the disk is busy.
7724+
7725+
#### Event: `'close'`
7726+
7727+
The `'close'` event is emitted when the stream is fully closed.
7728+
7729+
#### Event: `'drain'`
7730+
7731+
The `'drain'` event is emitted when the internal buffer has drained sufficiently
7732+
to allow continued writing.
7733+
7734+
#### Event: `'drop'`
7735+
7736+
The `'drop'` event is emitted when to maximal length is reached and that data
7737+
will not be written. The data that was dropped is passed as the first argument
7738+
to the event handle.
7739+
7740+
#### Event: `'error'`
7741+
7742+
The `'error'` event is emitted when an error occurs.
7743+
7744+
#### Event: `'finish'`
7745+
7746+
The `'finish'` event is emitted when the stream has been ended and all data has
7747+
been flushed to the underlying file.
7748+
7749+
#### Event: `'ready'`
7750+
7751+
The `'ready'` event is emitted when the stream is ready to accept writes.
7752+
7753+
#### Event: `'write'`
7754+
7755+
The `'write'` event is emitted when a write operation has completed. The number
7756+
of bytes written is passed as the first argument to the event handler.
7757+
7758+
#### `new fs.Utf8Stream([options])`
7759+
7760+
* `options` {Object}
7761+
* `append`: {boolean} Appends writes to dest file instead of truncating it.
7762+
**Default**: `true`.
7763+
* `contentMode`: {string} Which type of data you can send to the write
7764+
function, supported values are `'utf8'` or `'buffer'`. **Default**:
7765+
`'utf8'`.
7766+
* `dest`: {string} A path to a file to be written to (mode controlled by the
7767+
append option).
7768+
* `fd`: {number} A file descriptor, something that is returned by `fs.open()`
7769+
or `fs.openSync()`.
7770+
* `fs`: {Object} An object that has the same API as the `fs` module, useful
7771+
for mocking, testing, or customizing the behavior of the stream.
7772+
* `fsync`: {boolean} Perform a `fs.fsyncSync()` every time a write is
7773+
completed.
7774+
* `maxLength`: {number} The maximum length of the internal buffer. If a write
7775+
operation would cause the buffer to exceed `maxLength`, the data written is
7776+
dropped and a drop event is emitted with the dropped data
7777+
* `maxWrite`: {number} The maximum number of bytes that can be written;
7778+
**Default**: `16384`
7779+
* `minLength`: {number} The minimum length of the internal buffer that is
7780+
required to be full before flushing.
7781+
* `mkdir`: {boolean} Ensure directory for `dest` file exists when true.
7782+
**Default**: `false`.
7783+
* `mode`: {number|string} Specify the creating file mode (see `fs.open()`).
7784+
* `periodicFlush`: {number} Calls flush every `periodicFlush` milliseconds.
7785+
* `retryEAGAIN` {Function} A function that will be called when `write()`,
7786+
`writeSync()`, or `flushSync()` encounters an `EAGAIN` or `EBUSY` error.
7787+
If the return value is `true` the operation will be retried, otherwise it
7788+
will bubble the error. The `err` is the error that caused this function to
7789+
be called, `writeBufferLen` is the length of the buffer that was written,
7790+
and `remainingBufferLen` is the length of the remaining buffer that the
7791+
stream did not try to write.
7792+
* `err` {any} An error or `null`.
7793+
* `writeBufferLen` {number}
7794+
* `remainingBufferLen`: {number}
7795+
* `sync`: {boolean} Perform writes synchronously.
7796+
7797+
#### `utf8Stream.append`
7798+
7799+
* {boolean} Whether the stream is appending to the file or truncating it.
7800+
7801+
#### `utf8Stream.contentMode`
7802+
7803+
* {string} The type of data that can be written to the stream. Supported
7804+
values are `'utf8'` or `'buffer'`. **Default**: `'utf8'`.
7805+
7806+
#### `utf8Stream.destroy()`
7807+
7808+
Close the stream immediately, without flushing the internal buffer.
7809+
7810+
#### `utf8Stream.end()`
7811+
7812+
Close the stream gracefully, flushing the internal buffer before closing.
7813+
7814+
#### `utf8Stream.fd`
7815+
7816+
* {number} The file descriptor that is being written to.
7817+
7818+
#### `utf8Stream.file`
7819+
7820+
* {string} The file that is being written to.
7821+
7822+
#### `utf8Stream.flush(callback)`
7823+
7824+
* `callback` {Function}
7825+
* `err` {Error|null} An error if the flush failed, otherwise `null`.
7826+
7827+
Writes the current buffer to the file if a write was not in progress. Do
7828+
nothing if `minLength` is zero or if it is already writing.
7829+
7830+
#### `utf8Stream.flushSync()`
7831+
7832+
Flushes the buffered data synchronously. This is a costly operation.
7833+
7834+
#### `utf8Stream.fsync`
7835+
7836+
* {boolean} Whether the stream is performing a `fs.fsyncSync()` after every
7837+
write operation.
7838+
7839+
#### `utf8Stream.maxLength`
7840+
7841+
* {number} The maximum length of the internal buffer. If a write
7842+
operation would cause the buffer to exceed `maxLength`, the data written is
7843+
dropped and a drop event is emitted with the dropped data.
7844+
7845+
#### `utf8Stream.minLength`
7846+
7847+
* {number} The minimum length of the internal buffer that is required to be
7848+
full before flushing.
7849+
7850+
#### `utf8Stream.mkdir`
7851+
7852+
* {boolean} Whether the stream should ensure that the directory for the
7853+
`dest` file exists. If `true`, it will create the directory if it does not
7854+
exist. **Default**: `false`.
7855+
7856+
#### `utf8Stream.mode`
7857+
7858+
* {number|string} The mode of the file that is being written to.
7859+
7860+
#### `utf8Stream.periodicFlush`
7861+
7862+
* {number} The number of milliseconds between flushes. If set to `0`, no
7863+
periodic flushes will be performed.
7864+
7865+
#### `utf8Stream.reopen(file)`
7866+
7867+
* `file`: {string|Buffer|URL} A path to a file to be written to (mode
7868+
controlled by the append option).
7869+
7870+
Reopen the file in place, useful for log rotation.
7871+
7872+
#### `utf8Stream.sync`
7873+
7874+
* {boolean} Whether the stream is writing synchronously or asynchronously.
7875+
7876+
#### `utf8Stream.write(data)`
7877+
7878+
* `data` {string|Buffer} The data to write.
7879+
* Returns {boolean}
7880+
7881+
When the `options.contentMode` is set to `'utf8'` when the stream is created,
7882+
the `data` argument must be a string. If the `contentMode` is set to `'buffer'`,
7883+
the `data` argument must be a {Buffer}.
7884+
7885+
#### `utf8Stream.writing`
7886+
7887+
* {boolean} Whether the stream is currently writing data to the file.
7888+
7889+
#### `utf8Stream[Symbol.dispose]()`
7890+
7891+
Calls `utf8Stream.destroy()`.
7892+
77137893
### Class: `fs.WriteStream`
77147894
77157895
<!-- YAML

lib/fs.js

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,11 @@ let ReadFileContext;
165165
// monkeypatching.
166166
let FileReadStream;
167167
let FileWriteStream;
168+
let Utf8Stream;
169+
170+
function lazyLoadUtf8Stream() {
171+
Utf8Stream ??= require('internal/streams/fast-utf8-stream');
172+
}
168173

169174
// Ensure that callbacks run in the global context. Only use this function
170175
// for callbacks that are passed to the binding layer, callbacks that are
@@ -3329,6 +3334,11 @@ module.exports = fs = {
33293334
FileWriteStream = val;
33303335
},
33313336

3337+
get Utf8Stream() {
3338+
lazyLoadUtf8Stream();
3339+
return Utf8Stream;
3340+
},
3341+
33323342
// For tests
33333343
_toUnixTimestamp: toUnixTimestamp,
33343344
};

0 commit comments

Comments
 (0)