• Meteor
  • Tutorial

File Uploader Part 6: Streaming Uploads

We're getting pretty close to being able to use the Meteor file in an actual application. And in the last episode, we converted our read method on the Meteor file prototype to be able to read in files a specified number of bytes that a time. In other words, we are able to stream to read. And what we want to be able to do is to read in a certain number of bytes and then send those up to the server, have the server right those to a file, and when it returns read in the next set of bytes, and so on, until it completes.

And so in this episode, we're going to implement the streaming upload method. And so we have the upload method on the prototype of Meteor file so that we can call it on instances of Meteor file. And then I've also added it here on the Meteor file itself so it's more convenient to use in applications. So let's get started.

So let's start with the meat of this method. And then we'll go back, and we'll clean it up at the end. So the first thing I'm going to do is just create the self variable. And what we want to do is start off with all of our counters set in the right place, because we're going to iterate through this file, starting at 0 and working through the file length. And so let's create a method called rewind. And all that's going to do is set the data of the instance to null, reset start to zero, reset n to zero, and the same for bytes read and bytes uploaded.

And next, let's set the size property of the Meteor file instance equal to the file's total size. So the size property will be the total size of the file. That's where the size specified in options will be the chunk size, or the number of bytes that we want to upload at a given time.

Next, the trick here is that we want to read in a piece of the file and send it to the server. But we want to wait until the server returns before we read in the next set of bytes and upload those. So what I'm going to do is a trick. I'm going to create a function called read next. And it's just going to be an iterator function. And what this iterator function will do is it sort of works like a call back, where we can call it when we're ready to do the next iteration. And just so I don't forget, down at the bottom we'll initialize the iterator by calling read next. And also so I don't forget, we'll return this so that we can chain the upload method.

OK, inside the read next method, the first thing we'll do is we'll check whether or not the number of bytes uploaded is less than the size. So in other words, have we uploaded all the bytes or not? If we've uploaded all the bytes, in other words, the upload is done, we'll make sure that the callback is truthy. And if it is, we'll call it with no error and the Meteor file as the second parameter.

OK, so if we're not done, what we'd want to do is we want to start iterating over the files. So to do that, I'm going to call the read method that we just created in the last episode. And I'll pass the file that's the first parameter that was passed into this function. And then I'll just pass along the options that we got inside this method. And then I'll create a call back that I know the read method will call when it's done reading.

And when the read method completes, and it's done reading, what we want to do is to call the Meteor method. But instead of having it hard-coded this time, we allow the user to provide it as a name. So that's the second parameter of our upload function. And so what we can do is we'll call meteor.apply. And as the first parameter will pass the method-- so this will be the method name, like upload file.

And I'm going to use a trick here where I'm going to pass the Meteor file as the first parameter. And then I'm going to concatenate any params that were passed along in the options, or an empty array if the option ,params is falsey. And so that'll let a user called Upload and then pass additional parameters to the method through the options hash OK next I'm going to pass some options to the Meteor apply method. And I'm going to say wait true, and that's going to tell Meteor to wait for this method to return before allowing other methods to be called.

And then finally, my calllback function with an error as a parameter, if there is one. So if there's an error-- let's center this-- what we'll do is we'll say if there's an error and if the callback is defined, we'll go ahead and call our callback with the error as the first parameter. If there's an error and there's no callback defined, we'll just throw it.

Otherwise, what we want to do is let's increment the number of bytes that have been uploaded. And we can get that by what are the length of our current data. And then we'll call read next. And that will continue the iteration.

OK, we're almost done. There's just one more thing I'd like to do. I'd like to clean up this function a little bit. And so the first thing I'd like to do is inside of the read callback, it's possible that we could get an error on the read. I can't think of what would cause that. But maybe if you moved the file in the middle of the read, or something like that.

If there's an error, and there's a callback, what we'll do is we'll call our callback with the error as a parameter. Otherwise, if there's just an error, but no callback, we'll throw it. Otherwise, we'll do the Meteor apply.

OK, and next, we just need to come up here and let's deal with some different types of options. So I'm actually going to paste in some code here. But don't worry too much about it.

I'll explain it quickly. The first two if statements here, I'm just making sure that what we're passing in inherits from blob. Because not to be able to deal very well with other types. So I could do here check for methods that exist on the file or things like that. But what I'm just doing is making sure that it's something that inherits from blob.

And then the second thing I'm doing is making sure that the user passes in a method name, because this function really can't operate without a method name. So it's a required parameter. But the options are optional.

So what we're saying here is if the argument size is less than four, what we'll do is we'll just say that the user likely assigned the callback as the third parameter. And we'll just set the options to an empty hash. And so this requires that the user passes two parameters, at least. And then the third parameter can either be an options hash or the callback. And then the rest of it looks pretty good.

And finally down here in the Meteor file upload method, we're just going to proxy to a new Meteor file, and call it the upload method, passing along all of these parameters. OK, so we should be good to go on the streaming upload. And next up, what we need to do is to implement the streaming right on the server side.