Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream this.push and spread operator does not work #26582

Closed
akaNightmare opened this issue Mar 11, 2019 · 2 comments
Closed

Stream this.push and spread operator does not work #26582

akaNightmare opened this issue Mar 11, 2019 · 2 comments
Labels
invalid Issues and PRs that are invalid. stream Issues and PRs related to the stream subsystem.

Comments

@akaNightmare
Copy link

  • Version: v11.10.0
  • Platform: Darwin Ivans-MacBook-Pro.local 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64

Hello, I have written a simple Transform stream to getting some extended user info:

import { Transform } from 'stream';
import { getUsersPublicInfo } from '../../user/services';

const USERS_BATCH_SIZE = 100;

/**
 * Class (extended from Transform stream)
 * for getting minimum user info for diff
 * types of reports. Returns `id`, `email`
 * and `name` in a stream as an object.
 */
export class FillUserTransform extends Transform {
    constructor(options) {
        super({
            readableObjectMode: true,
            writableObjectMode: true,
            ...(options || {}),
        });
        this._userIds = [];
    }

    /**
     * @param {{user_id: string}} chunk
     * @param {string} encoding
     * @param {Function} cb
     * @return {*}
     * @private
     */
    _transform(chunk, encoding, cb) {
        this._userIds.push(chunk.user_id);
        if (this._userIds.length >= USERS_BATCH_SIZE) {
            this._writeUserInfo(cb);
        } else {
            return void cb();
        }
    }

    _flush(cb) {
        if (this._userIds.length > 0) {
            this._writeUserInfo(cb);
        } else {
            return void cb();
        }
    }

    /**
     * @param {Function} cb
     * @return {*}
     * @private
     */
    _writeUserInfo(cb) {
        this.pause();
        getUsersPublicInfo(this._userIds)
            .then(users => {
                if (users.length > 0) {
                    this.push(...users);
                }
                this._userIds.length = 0;
                this.resume();
                cb();
            })
            .catch(err => {
                this._userIds.length = 0;
                this.resume();
                cb(err);
            });
    }
}

but this.push(...users); does not work as expected, replaced spread operator with forEach - works

users.forEach(user => {
    this.push(user);
});
@lpinca
Copy link
Member

lpinca commented Mar 11, 2019

I think it works as expected. readable.push() takes one chunk at max.

@lpinca lpinca added the stream Issues and PRs related to the stream subsystem. label Mar 11, 2019
@bnoordhuis
Copy link
Member

Closing as answered.

@bnoordhuis bnoordhuis added the invalid Issues and PRs that are invalid. label Mar 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid Issues and PRs that are invalid. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Development

No branches or pull requests

3 participants