Skip to content

stream: do not chunk strings and Buffer in Readable.from. #30912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

stream: do not chunk strings and Buffer in Readable.from. #30912

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/api/stream.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1642,6 +1642,10 @@ readable.on('data', (chunk) => {
});
```

Calling `Readable.from(string)` or `Readable.from(buffer)` will not have
the strings or buffers be iterated to match the other streams semantics
for performance reasons.

## API for Stream Implementers

<!--type=misc-->
Expand Down
12 changes: 12 additions & 0 deletions lib/internal/streams/from.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,25 @@ const {
Symbol,
SymbolIterator
} = primordials;
const { Buffer } = require('buffer');

const {
ERR_INVALID_ARG_TYPE
} = require('internal/errors').codes;

function from(Readable, iterable, opts) {
let iterator;
if (typeof iterable === 'string' || iterable instanceof Buffer) {
Copy link
Member

@ronag ronag Dec 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check for Stream._isUint8Array as well?

Copy link
Member

@addaleax addaleax Dec 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would only be okay if we turn off object mode, I think

return new Readable({
objectMode: true,
Copy link
Member

@ronag ronag Dec 12, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, Not sure of the purpose here, but wouldn't objectMode: false make more sense here as a default? Or is that breaking?

Copy link
Member Author

@mcollina mcollina Dec 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep it consistent with the rest. Also, it would change the encoding, so possibly it's not a good idea.

...opts,
read() {
this.push(iterable);
this.push(null);
}
});
}

if (iterable && iterable[Symbol.asyncIterator])
iterator = iterable[Symbol.asyncIterator]();
else if (iterable && iterable[SymbolIterator])
Expand Down
13 changes: 12 additions & 1 deletion test/parallel/test-readable-from.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,23 @@ async function toReadablePromises() {
async function toReadableString() {
const stream = Readable.from('abc');

const expected = ['a', 'b', 'c'];
const expected = ['abc'];

for await (const chunk of stream) {
strictEqual(chunk, expected.shift());
}
}

async function toReadableBuffer() {
const stream = Readable.from(Buffer.from('abc'));

const expected = ['abc'];

for await (const chunk of stream) {
strictEqual(chunk.toString(), expected.shift());
}
}

async function toReadableOnData() {
async function* generate() {
yield 'a';
Expand Down Expand Up @@ -154,6 +164,7 @@ Promise.all([
toReadableSyncIterator(),
toReadablePromises(),
toReadableString(),
toReadableBuffer(),
toReadableOnData(),
toReadableOnDataNonObject(),
destroysTheStreamWhenThrowing(),
Expand Down