Skip to content

Commit 3a8cf24

Browse files
jasnelladuh95
authored andcommitted
fs: port SonicBoom module to fs module as Utf8Stream
As a first step to porting portions of the pino structured logger into the runtime, this commit ports the SonicBoom module to the fs module as Utf8Stream. This is a faithful port of the SonicBoom module with some modern updates, such as converting to a Class and using Symbol.dispose. The bulk of the implementation is unchanged from the original. PR-URL: #58897 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Vinícius Lourenço Claro Cardoso <[email protected]>
1 parent a8f381a commit 3a8cf24

18 files changed

+2890
-0
lines changed

doc/api/fs.md

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7642,6 +7642,186 @@ added:
76427642
76437643
Type of file system.
76447644
7645+
### Class: `fs.Utf8Stream`
7646+
7647+
<!-- YAML
7648+
added: REPLACEME
7649+
-->
7650+
7651+
> Stability: 1 - Experimental
7652+
7653+
An optimized UTF-8 stream writer that allows for flushing all the internal
7654+
buffering on demand. It handles `EAGAIN` errors correctly, allowing for
7655+
customization, for example, by dropping content if the disk is busy.
7656+
7657+
#### Event: `'close'`
7658+
7659+
The `'close'` event is emitted when the stream is fully closed.
7660+
7661+
#### Event: `'drain'`
7662+
7663+
The `'drain'` event is emitted when the internal buffer has drained sufficiently
7664+
to allow continued writing.
7665+
7666+
#### Event: `'drop'`
7667+
7668+
The `'drop'` event is emitted when to maximal length is reached and that data
7669+
will not be written. The data that was dropped is passed as the first argument
7670+
to the event handle.
7671+
7672+
#### Event: `'error'`
7673+
7674+
The `'error'` event is emitted when an error occurs.
7675+
7676+
#### Event: `'finish'`
7677+
7678+
The `'finish'` event is emitted when the stream has been ended and all data has
7679+
been flushed to the underlying file.
7680+
7681+
#### Event: `'ready'`
7682+
7683+
The `'ready'` event is emitted when the stream is ready to accept writes.
7684+
7685+
#### Event: `'write'`
7686+
7687+
The `'write'` event is emitted when a write operation has completed. The number
7688+
of bytes written is passed as the first argument to the event handler.
7689+
7690+
#### `new fs.Utf8Stream([options])`
7691+
7692+
* `options` {Object}
7693+
* `append`: {boolean} Appends writes to dest file instead of truncating it.
7694+
**Default**: `true`.
7695+
* `contentMode`: {string} Which type of data you can send to the write
7696+
function, supported values are `'utf8'` or `'buffer'`. **Default**:
7697+
`'utf8'`.
7698+
* `dest`: {string} A path to a file to be written to (mode controlled by the
7699+
append option).
7700+
* `fd`: {number} A file descriptor, something that is returned by `fs.open()`
7701+
or `fs.openSync()`.
7702+
* `fs`: {Object} An object that has the same API as the `fs` module, useful
7703+
for mocking, testing, or customizing the behavior of the stream.
7704+
* `fsync`: {boolean} Perform a `fs.fsyncSync()` every time a write is
7705+
completed.
7706+
* `maxLength`: {number} The maximum length of the internal buffer. If a write
7707+
operation would cause the buffer to exceed `maxLength`, the data written is
7708+
dropped and a drop event is emitted with the dropped data
7709+
* `maxWrite`: {number} The maximum number of bytes that can be written;
7710+
**Default**: `16384`
7711+
* `minLength`: {number} The minimum length of the internal buffer that is
7712+
required to be full before flushing.
7713+
* `mkdir`: {boolean} Ensure directory for `dest` file exists when true.
7714+
**Default**: `false`.
7715+
* `mode`: {number|string} Specify the creating file mode (see `fs.open()`).
7716+
* `periodicFlush`: {number} Calls flush every `periodicFlush` milliseconds.
7717+
* `retryEAGAIN` {Function} A function that will be called when `write()`,
7718+
`writeSync()`, or `flushSync()` encounters an `EAGAIN` or `EBUSY` error.
7719+
If the return value is `true` the operation will be retried, otherwise it
7720+
will bubble the error. The `err` is the error that caused this function to
7721+
be called, `writeBufferLen` is the length of the buffer that was written,
7722+
and `remainingBufferLen` is the length of the remaining buffer that the
7723+
stream did not try to write.
7724+
* `err` {any} An error or `null`.
7725+
* `writeBufferLen` {number}
7726+
* `remainingBufferLen`: {number}
7727+
* `sync`: {boolean} Perform writes synchronously.
7728+
7729+
#### `utf8Stream.append`
7730+
7731+
* {boolean} Whether the stream is appending to the file or truncating it.
7732+
7733+
#### `utf8Stream.contentMode`
7734+
7735+
* {string} The type of data that can be written to the stream. Supported
7736+
values are `'utf8'` or `'buffer'`. **Default**: `'utf8'`.
7737+
7738+
#### `utf8Stream.destroy()`
7739+
7740+
Close the stream immediately, without flushing the internal buffer.
7741+
7742+
#### `utf8Stream.end()`
7743+
7744+
Close the stream gracefully, flushing the internal buffer before closing.
7745+
7746+
#### `utf8Stream.fd`
7747+
7748+
* {number} The file descriptor that is being written to.
7749+
7750+
#### `utf8Stream.file`
7751+
7752+
* {string} The file that is being written to.
7753+
7754+
#### `utf8Stream.flush(callback)`
7755+
7756+
* `callback` {Function}
7757+
* `err` {Error|null} An error if the flush failed, otherwise `null`.
7758+
7759+
Writes the current buffer to the file if a write was not in progress. Do
7760+
nothing if `minLength` is zero or if it is already writing.
7761+
7762+
#### `utf8Stream.flushSync()`
7763+
7764+
Flushes the buffered data synchronously. This is a costly operation.
7765+
7766+
#### `utf8Stream.fsync`
7767+
7768+
* {boolean} Whether the stream is performing a `fs.fsyncSync()` after every
7769+
write operation.
7770+
7771+
#### `utf8Stream.maxLength`
7772+
7773+
* {number} The maximum length of the internal buffer. If a write
7774+
operation would cause the buffer to exceed `maxLength`, the data written is
7775+
dropped and a drop event is emitted with the dropped data.
7776+
7777+
#### `utf8Stream.minLength`
7778+
7779+
* {number} The minimum length of the internal buffer that is required to be
7780+
full before flushing.
7781+
7782+
#### `utf8Stream.mkdir`
7783+
7784+
* {boolean} Whether the stream should ensure that the directory for the
7785+
`dest` file exists. If `true`, it will create the directory if it does not
7786+
exist. **Default**: `false`.
7787+
7788+
#### `utf8Stream.mode`
7789+
7790+
* {number|string} The mode of the file that is being written to.
7791+
7792+
#### `utf8Stream.periodicFlush`
7793+
7794+
* {number} The number of milliseconds between flushes. If set to `0`, no
7795+
periodic flushes will be performed.
7796+
7797+
#### `utf8Stream.reopen(file)`
7798+
7799+
* `file`: {string|Buffer|URL} A path to a file to be written to (mode
7800+
controlled by the append option).
7801+
7802+
Reopen the file in place, useful for log rotation.
7803+
7804+
#### `utf8Stream.sync`
7805+
7806+
* {boolean} Whether the stream is writing synchronously or asynchronously.
7807+
7808+
#### `utf8Stream.write(data)`
7809+
7810+
* `data` {string|Buffer} The data to write.
7811+
* Returns {boolean}
7812+
7813+
When the `options.contentMode` is set to `'utf8'` when the stream is created,
7814+
the `data` argument must be a string. If the `contentMode` is set to `'buffer'`,
7815+
the `data` argument must be a {Buffer}.
7816+
7817+
#### `utf8Stream.writing`
7818+
7819+
* {boolean} Whether the stream is currently writing data to the file.
7820+
7821+
#### `utf8Stream[Symbol.dispose]()`
7822+
7823+
Calls `utf8Stream.destroy()`.
7824+
76457825
### Class: `fs.WriteStream`
76467826
76477827
<!-- YAML

lib/fs.js

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,11 @@ let ReadFileContext;
167167
// monkeypatching.
168168
let FileReadStream;
169169
let FileWriteStream;
170+
let Utf8Stream;
171+
172+
function lazyLoadUtf8Stream() {
173+
Utf8Stream ??= require('internal/streams/fast-utf8-stream');
174+
}
170175

171176
function showTruncateDeprecation() {
172177
if (truncateWarn) {
@@ -3344,6 +3349,11 @@ module.exports = fs = {
33443349
FileWriteStream = val;
33453350
},
33463351

3352+
get Utf8Stream() {
3353+
lazyLoadUtf8Stream();
3354+
return Utf8Stream;
3355+
},
3356+
33473357
// For tests
33483358
_toUnixTimestamp: toUnixTimestamp,
33493359
};

0 commit comments

Comments
 (0)