Ending a Stream Without Consuming Its Data

I ran into an issue recently while writing a node module. My process would sometimes complete its work, but wouldn’t end. After a bit of debugging, I figured out what was happening.

I had implemented a handler for the HTTP “location” header. If a response came back with that header, I would ignore that response and go fetch the new location instead. Unfortunately, whenever I did that, the process would finish fetching the new location, and then just hang there for about a minute bofore exiting. The reason this happened was because I wasn’t properly handling the first response.

Here’s an oversimplified explanation of what happens. This is from my limited understanding of Node internals, so the details may be slighly off, but the overall idea is correct.

When you use the “http” module to make a request in Node, the response it gives you is a stream. If you don’t do anything to consume that stream’s data, Node doesn’t exit because there’s still work to be done (consuming the stream). In order to get Node to exit without something clunky like process.exit(), you just need to switch the stream to “flowing” mode.

1
2
3
4
5
6
7
8
9
10
11
12
var http = require('http');

var responseHandler = function(response) {
  if (response.headers.location) {
    http.get(response.headers.location, responseHandler);
    response.resume();
  } else {
    response.pipe(process.stdout);
  }
};

http.get(process.argv[2], responseHandler);

Notice on line 6 how I’ve called response.resume(). This switches the response stream to “flowing” mode. According to the streams documentation, this is the way to finish a stream if you don’t care about consuming its data.

I am now accepting new clients for part-time consulting and software development projects. Learn more

I haven't configured comments for this blog, but if you want to get in touch, you can find me on Twitter