Syncify with a Subprocess.

Hi Everyone,

I just wanted to open a discussion about something that I’ve been working on, before I work too hard on it.

I’ve talked about this before, but here’s my idea:

• Because of the way Node works, it’s a lot easier to write things asyncly.
• But we want shelljs commands to be sync.
• Therefore, it would be nicer if we could syncify things,
• However, this doesn’t work:

function syncSomething(arg1) {
    var ret = null, err = null, done = false;
    something(arg1, function (err, ret) {
        // We will never get here, because the event loop will never be clear, so the callback can never start.
        ret = _ret;
        err = _err;
        done = true;
    });
    while(!done) continue;
    if (err) throw err;
    return ret;
}

• So, the only way to syncify (at least without a native extension) something is to make a second event loop.
• The only way to do that is to start a second process.
• So, what we could do is this:

1. After all the argument parsing, common.wrap could do this:

// ...
if (callback || options.cp === false) {
    func.apply(null, args.concat([callback]));
} else {
    if (!this.cp) this.cp = new ChildProcess();
    return this.cp.cmd(this.name, args);
}

1. ChildProcess.prototype.cmd could do this:

ChildProcess.prototype.cmd = function(cmd, args) {
    var id = Math.random().toString();
    write({ type: 'cmd', cmd, args, id }); // I've already written read/write, they work, so I won't go into it here.
    var ret, err, done;
    while (!done) {
        var events = read();
        for (var i = 0; i < events.length; ++i) {
            if (events[i].type === 'cmd' && events[i].id === id) {
                ret = events[i].ret;
                err = events[i].err;
                done = true;
            }
        }
    }
    if (err) throw err;
    return ret;
}

• But since I realized that this was going to basically be a rewrite of shelljs, I thought we should discuss it first.
• Also, @levithomason setup a really awesome build and lint system for shelljs/shx, so if we’re going to re-write it, I think we might want to get something similar. (@levithomason, would you be interested/willing in working this?)

Things this would fix (@shelljs/contributors, feel free to edit this and add stuff):

• #412, #186 - We can spawn the subprocess with { stdio: 'inherit' }, and then it can run exec commands the same way, since it doesn’t need to do the weird thing that exec currently does. That way stdin/stdout/stderr are tty’s.
• #2 - It’s easier to have an async version if you don’t have to maintain a sync version.
• #3 - We can use packages that are stream-only.
• #5, #167 - We don’t have to rely on a hacky wait loop.
• #7
• Maybe #158 - I think this might be caused by exec’s wait hack, or by improper cleanup.
• #248

So, @shelljs/contributors and @shelljs/shx, what are your thoughts on this?

[levithomason]

Mon 28th @ 1 pm 4 PM PST

Plan next steps https://appear.in/shelljs

Rough agenda:

• shipping 0.7, what’s left?
• existing issues/PRs cleanup plan
• sync/async plan
• shx, what’s next?

[nfischer]

I’m going to close this, since it would take too much effort to rewrite ShellJS. If someone really wants this, I think it’s best done as a fork of ShellJS.