• Node.js
  • Tutorial

Uploading a File with Node.js and Busboy

In a previous tutorial, I showed you how a browser can upload a file to a server using HTTP.

Here's an example HTTP request as it's seen on the server. Down below we have the body of the request, and we need something running on the server that's going to parse this body and turn it into objects or files that we can work with in our code. In the rest of this tutorial, I'm going to show you how we can upload files to the node JS framework using an NPM package called busboy. You can find busboy on the NPM website or on GitHub, and from its description you can see that it's a library for parsing HTML form data for node.js. And so that text body that you just saw in the HTTP request can be parsed by busboy and turned into JavaScript objects that we can use in our code.

A quick way you can add busboy to your project is to type npm install busboy, and that's going to put it inside of the node modules folder of the current directory. If you're following along with this code, I have a package.json file, which includes all the dependencies that are required for this project. So I'll open this up in my VIM editor and go into package.json. And notice that the busboy dependency is declared here. And I want to use Version 0.2.11.

Then what you can do is type npm install, and that will install all the dependencies that are listed in the package.json file. And to start up this server you can type npm start. If you open up the app.html file, notice I have a form here with an encoding type of multi-part form data, and then I have two inputs-- one for the title and the other for the actual file that we're going to upload. And then a third input to submit the form.

I've opened up the server.js file, and inside of here I have a very simple HTTP server. Let me walk you through it. We start off by creating the server using the HTTP module from node. Then we pass a callback function that's going to be passed a request object and a response object as parameters. Then I store a reference to the method, whether it be a get or a post or a patch or any of the other HTTP methods, and then a space and then the actual path or URL that's being requested. Then in a switch statement I just check to see what the URL is and what the verb is, and for each one I handle the response appropriately.

So here's our main app being returned here, and in our post upload, this is where we're going to deal with the file upload of our form. I'm working directly with node.js here instead of using a higher level framework like Meteor or Express so that you can see exactly what's happening with the request and the response.

Now, before we move on to busboy, I wanted to point out a mistake that people commonly make, which is to check the body property of the request and hope to find the body of the HTTP response in this property. Now, the problem with that is that the HTTP body is treated by node.js as a stream, and that means that it can come off the wire over time, not all at once. So for example, in this HTTP body here, the browser might send up this part of the form first and then a couple milliseconds later send up the rest of it. And if it's a big giant file, you can imagine the file being sent some number of kilobytes at a time. So another way to say this is that the body of the HTTP request has to be read in asynchronously.

Let's start off by requiring busboy, which is going to parse this HTTP form for us. I've added the busboy module on line three by just requiring busboy and assigning it to the variable busboy with a capital B. Inside of my route handler here for post upload, I need to create a new instance of busboy that I'll assign to a variable named busboy. And I can do that by saying, new busboy, and then just passing in the request as a parameter.

Next, down at the bottom of the handler I'm going to call a special method of the request object called pipe. What this is going to do is to take the request and pipe its stream into another object which is going to read from this stream. And I'll cover pipes in more detail in a different tutorial, but for now you can just call this pipe method, and what we need to do is to pass it, the busboy instance that we just created.

The way we interact with busboy is by responding to different events that it will emit. For example, it emits a finish event which will get called when the busboy is completely done parsing the request. And inside of that event handler, we can paste in our previous code that sends back a 303 HTTP status code, along with the new location that we want the browser to redirect to.

The next event that we can attach a handler for is the on file event. The file event handler is going to take a couple of parameters. The first is the name of the field as it was named on the form. The second is going to be a file object, so this is going to be a stream that we can read from and write it anywhere, like to standard out or to the file system. The file name is the actual original file name, the encoding is what was used to encode the file, and the MIME type is the type of file like, for example, a text file or MP4.

I've logged each one of these parameters to the console so that we can see what they look like, and we'll need to restart the node server since we've made changes to the server. And then, let's head over to the browser. I'll leave title field blank since we're not looking at that quite yet, and I'll choose the upload.txt file and click the Upload button. Here's the parameters that we printed out, and notice that the file is an object and it's a special kind of object-- a stream object-- that we can read from. So that allows us to read from this stream and then write that stream somewhere else, like to disk or even to standard output.

Let's take the contents of the file and print it out to the log so that we can see it right in the console. The way we can do that is by just piping the output. So I can call the pipe method on the file and then give it a place to pipe to. And so we're going to send it to the processes standard out. That'll be to the console so we can see it. Now notice that we're calling the pipe method just like we are on the request. So the request is a streamable object and so is this file, and that's why we're able to do this.

Once again, let's restart the server since we've made changes to the server file and then upload a new file to see what happens. I'll select the same text file as before and upload it to the server. And this time, we're taking the contents of that text file and just printing it right out to standard output, which is the same thing as the log.

Now usually, we don't want to take the contents of an uploaded file and just write it out to standard output. We want to save the file to disk on the server. So to do that I'm going to add a different module here. Another module called path. That's going to let us construct a path to our current location. And then inside of the callback, let's create a variable called pathname, and I'll set that equal to path.join. And we're going to grab the current directory of the currently executing script. That's what this __dirname gives us, and then I'm going to join that with a new file name uploaded.txt. So this will give us a full path to this uploaded.txt file.

Next, I'm going to create a write stream object by calling the fs.createWriteStream method and passing it a path name. And finally, I can pass that right stream object in as a parameter to the file that pipe method. So one of the benefits of using these pipe methods is we can take the file a little bit at a time. We don't have to read the entire file into memory. And with each little bit that we get we can write that chunk to the disk one piece at a time. So we're taking a read stream and piping it to a write stream.

Restart the server again since we've made a change and head over to the browser. I'll select the same text file and upload it, and notice the file has been saved to disk right here where in the same directory as the server.js file is running. And if we want to make sure that the contents of the file were correctly written, we can take a look at the file using the cat commander. Just open it up in your favorite text editor. And there's the Hello World text.

So far we've only grab the file off of the HTTP post, but we haven't looked at any of the other fields that were set up, like our title. To do that, I can add another event handler called field. So I'll call busboy.on. And the callback function-- this time we'll take the field name and then the value for that field. And you can do whatever you'd like with these fields, but let's just print it to the log. And now you've seen how we can use the busboy module to read in a file and fields from an HTTP form post.

Now, if you're using the connect middleware package that's used in Media or Express in another high level framework and you want to plug-in busboy as a middleware package, just make sure to call the next method, once you're done here, to pass on control to the next middleware. And the place to do that would be inside of this finished callback instead of ending the response. I'll give a demo of how to do that in a separate tutorial.