Skip to content

Commit da471d7

Browse files
committed
fs: port SonicBoom module to fs module as FastUtf8Stream
As a first step to porting portions of the pino structured logger into the runtime, this commit ports the SonicBoom module to the fs module as FastUtf8Stream. This is a faithful port of the SonicBoom module with some modern updates, such as converting to a Class and using Symbol.dispose. The bulk of the implementation is unchanged from the original.
1 parent 9501c86 commit da471d7

File tree

6 files changed

+2372
-0
lines changed

6 files changed

+2372
-0
lines changed

doc/api/fs.md

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6909,6 +6909,147 @@ changes:
69096909
69106910
The path to the parent directory of the file this {fs.Dirent} object refers to.
69116911
6912+
### Class: `fs.FastUtf8Stream`
6913+
6914+
<!-- YAML
6915+
added: REPLACEME
6916+
-->
6917+
6918+
> Stability: 1 - Experimental
6919+
6920+
An optimized UTF-8 stream writer that allows for flushing all the internal
6921+
buffering on demand. It handles `EAGAIN` errors correctly, allowing for
6922+
customization, for example, by dropping content if the disk is busy.
6923+
6924+
#### `new fs.FastUtf8Stream([options])`
6925+
6926+
* `options` {Object}
6927+
* `fd`: {number} A file descriptor, something that is returned by `fs.open()`
6928+
or `fs.openSync()`.
6929+
* `dest`: {string} A path to a file to be written to (mode controlled by the
6930+
append option).
6931+
* `minLength`: {number} The minimum length of the internal buffer that is
6932+
required to be full before flushing.
6933+
* `maxLength`: {number} The maximum length of the internal buffer. If a write
6934+
operation would cause the buffer to exceed `maxLength`, the data written is
6935+
dropped and a drop event is emitted with the dropped data
6936+
* `maxWrite`: {number} The maximum number of bytes that can be written;
6937+
**Default**: `16384`
6938+
* `periodicFlush`: {number} Calls flush every `periodicFlush` milliseconds.
6939+
* `sync`: {boolean} Perform writes synchronously.
6940+
* `fsync`: {boolean} Perform a `fs.fsyncSync()` every time a write is
6941+
completed.
6942+
* `append`: {boolean} Appends writes to dest file instead of truncating it.
6943+
**Default**: `true`.
6944+
* `mode`: {number|string} Specify the creating file mode (see `fs.open()`).
6945+
* `contentMode`: {string} Which type of data you can send to the write
6946+
function, supported values are `'utf8'` or `'buffer'`. **Default**:
6947+
`'utf8'`.
6948+
* `mkdir`: {boolean} Ensure directory for `dest` file exists when true.
6949+
**Default**: `false`.
6950+
* `retryEAGAIN` {Function} A function that will be called when `write()`,
6951+
`writeSync()`, or `flushSync()` encounters an `EAGAIN` or `EBUSY` error.
6952+
If the return value is `true` the operation will be retried, otherwise it
6953+
will bubble the error. The `err` is the error that caused this function to
6954+
be called, `writeBufferLen` is the length of the buffer that was written,
6955+
and `remainingBufferLen` is the length of the remaining buffer that the
6956+
stream did not try to write.
6957+
* `err` {any} An error or `null`.
6958+
* `writeBufferLen` {number}
6959+
* `remainingBufferLen`: {number}
6960+
6961+
#### `fastUtf8Stream.append`
6962+
6963+
* {boolean} Whether the stream is appending to the file or truncating it.
6964+
6965+
#### `fastUtf8Stream.contentMode`
6966+
6967+
* {string} The type of data that can be written to the stream. Supported
6968+
values are `'utf8'` or `'buffer'`. **Default**: `'utf8'`.
6969+
6970+
#### `fastUtf8Stream.destroy()`
6971+
6972+
Close the stream immediately, without flushing the internal buffer.
6973+
6974+
#### `fastUtf8Stream.end()`
6975+
6976+
Close the stream gracefully, flushing the internal buffer before closing.
6977+
6978+
#### `fastUtf8Stream.fd`
6979+
6980+
* {number} The file descriptor that is being written to.
6981+
6982+
#### `fastUtf8Stream.file`
6983+
6984+
* {string|Buffer|URL} The file that is being written to.
6985+
6986+
#### `fastUtf8Stream.flush(callback)`
6987+
6988+
* `callback` {Function}
6989+
* `err` {Error|null} An error if the flush failed, otherwise `null`.
6990+
6991+
Writes the current buffer to the file if a write was not in progress. Do
6992+
nothing if `minLength` is zero or if it is already writing.
6993+
6994+
#### `fastUtf8Stream.flushSync()`
6995+
6996+
Flushes the buffered data synchronously. This is a costly operation.
6997+
6998+
#### `fastUtf8Stream.fsync`
6999+
7000+
* {boolean} Whether the stream is performing a `fs.fsyncSync()` after every
7001+
write operation.
7002+
7003+
#### `fastUtf8Stream.maxLength`
7004+
7005+
* {number} The maximum length of the internal buffer. If a write
7006+
operation would cause the buffer to exceed `maxLength`, the data written is
7007+
dropped and a drop event is emitted with the dropped data.
7008+
7009+
#### `fastUtf8Stream.minLength`
7010+
7011+
* {number} The minimum length of the internal buffer that is required to be
7012+
full before flushing.
7013+
7014+
#### `fastUtf8Stream.mkdir`
7015+
7016+
* {boolean} Whether the stream should ensure that the directory for the
7017+
`dest` file exists. If `true`, it will create the directory if it does not
7018+
exist. **Default**: `false`.
7019+
7020+
#### `fastUtf8Stream.mode`
7021+
7022+
* {number|string} The mode of the file that is being written to.
7023+
7024+
#### `fastUtf8Stream.periodicFlush`
7025+
7026+
* {number} The number of milliseconds between flushes. If set to `0`, no
7027+
periodic flushes will be performed.
7028+
7029+
#### `fastUtf8Stream.reopen(file)`
7030+
7031+
* `file`: {string|Buffer|URL} A path to a file to be written to (mode
7032+
controlled by the append option).
7033+
7034+
Reopen the file in place, useful for log rotation.
7035+
7036+
#### `fastUtf8Stream.sync`
7037+
7038+
* {boolean} Whether the stream is writing synchronously or asynchronously.
7039+
7040+
#### `fastUtf8Stream.write(data)`
7041+
7042+
* `data` {string|Buffer} The data to write.
7043+
* Returns {boolean}
7044+
7045+
#### `fastUtf8Stream.writing`
7046+
7047+
* {boolean} Whether the stream is currently writing data to the file.
7048+
7049+
#### `fastUtf8Stream[Symbol.dispose]()`
7050+
7051+
Calls `fastUtf8Stream.destroy()`.
7052+
69127053
### Class: `fs.FSWatcher`
69137054
69147055
<!-- YAML

lib/fs.js

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,11 @@ let ReadFileContext;
164164
// monkeypatching.
165165
let FileReadStream;
166166
let FileWriteStream;
167+
let FastUtf8Stream;
168+
169+
function lazyLoadFastUtf8Stream() {
170+
FastUtf8Stream ??= require('internal/streams/fast-utf8-stream');
171+
}
167172

168173
// Ensure that callbacks run in the global context. Only use this function
169174
// for callbacks that are passed to the binding layer, callbacks that are
@@ -3297,6 +3302,11 @@ module.exports = fs = {
32973302
FileWriteStream = val;
32983303
},
32993304

3305+
get FastUtf8Stream() {
3306+
lazyLoadFastUtf8Stream();
3307+
return FastUtf8Stream;
3308+
},
3309+
33003310
// For tests
33013311
_toUnixTimestamp: toUnixTimestamp,
33023312
};

0 commit comments

Comments
 (0)