• Meteor

What is a Fiber?

From the class:  Meteor Fibers and Dynamics

I'm going to show you what a fiber is in this video, and we'll play around with it. But before we do that, I want to motivate why it is that we have fibers in the first place. So we've already talked about the fact that we get concurrent I/O by usually just calling some I/O function like a connect to Mongo, which makes a TCP connection to Mongo, and then passing in a callback function, which gets the result.

And if we want to, we can do some work after we make that connect call and that'll execute right away. And then once we have the connection, this callback function will get called. But a lot of the times, what we want to do is process or do something with the result right away. So instead of doing any other work, we just want to wait until we're connected to the database.

And then in this example, once we're connected we want to insert some documents. And then once we're done inserting documents, update the document. And then once we're done with that, do this. And then all the way down this chain here until finally, in this example, the database is closed.

So we just keep nesting and nesting and nesting. And there's a name for this. It's called callback hell. You might have heard this term. And what it basically means is that this is starting to get unruly. It looks strange to have all of these callbacks nested so far, so deeply. And it makes the code a bit hard to read.

So the primary purpose of a fiber is to allow us to write a synchronous style of code. So instead of passing in a callback like this, we just get a result returned. And we want to be able to do that without blocking other code that could potentially run at the same time. So let me give you an example in a drawing here before we go over to the code and start playing with a fiber.

Let's say that we have a request from a browser to our web server. So I'll just call this R1. And in the first request, what it's going to do is connect to the database. So we'll connect to Mongo here. And then it will make a Find call. So we're going to find a particular document and then return that document.

So for each one of these steps it's an asynchronous I/O. So we can make the connection, call, and then just run some other code until we have it, because this is just making a TCP call to Mongo. And the same thing with Find.

But within this request, we don't want to call the Find function until we're connected. And so instead of nesting all these, I'd just list to be able to write the code in a very simple synchronous fashion. But while we're executing this function in I/O, we could process another request.

So we'll say this is R2. And maybe R2 does the same thing. So it connects first and then it makes a Find call and then it returns. And for each one of these, while we're connecting to Mongo, we can go ahead and process the next request. So I'd like to have some ability to pause this execution until we have our result from connect, and then continue.

But while we're pausing just this, we want to be able to run future requests. So let me show you what I mean. We'll write some code together.

I've created a simple Meteor application and it has a server folder with an app.js file in it. So we're just going to write server code for this example. And in the right, I fired up the Node Inspector Debugger, and I'll show you how to start Meteor so that we can use this to debug on the server.

So we'll type meteor to start up the server, but I'm going to pass it a flag called debug port. And the debug port we want to be, I'll just say 8080. It can be any port you'd like. And when we do that, Meteor will start up, but it will automatically attach to Node Inspector and that way we can debug over here in the Chrome Debug Tools, just like we can in a normal browser session. So if we refresh this, it'll drop us in a break point and we can press play and inspect code on the server, which is really useful.

I've written some code and dropped us into the debugger. And what we're going to do in this code is just call a function, this function one here. And inside of that, we'll print out to the log that we're in function one and then call the next function and so on, all the way up until we get to three. And then we'll unwind the stack, all the way down until we finally print out this last line.

Now, as we step through this, keep an eye on the call stack over here on the right. Watch as it grows and as it shrinks. We'll step through this code together. And notice when I call into function one, the call stack grows and the first function is pushed onto the stack. And then when we call into function two, it gets pushed on, and so on until we get all the way up to function three.

And then as these functions start to return, they get popped off of the call stack. And so now we're back down to two and one, and then down to one, and finally we get down to this console log line. And if we press play, it'll print that it's OK to run now.

Now, let's say at some point in the execution of our function, we want to pause. And let's say that when we get to function two, for example, we want to just pause here. And then at some point in the future, resume execution exactly where we left off with the call stack saved and the exact point in execution or the instruction point are saved as well.

That's what fibers allow us to do. And let me show you first what a fiber is, and then later we'll look into how we can use it to get rid of the callback problem. So up at the top here, to get access to the fiber object, I'll create a variable. And we use the Meteor's NPM require to require it. To that'll give us access to fibers on the server.

Now a fiber, to create one, we can just create a variable here. I'll call it my fiber. So to create a new fiber, we call the fiber function and then we pass it a function that we want to execute. So in this case, we're going to run the function called one.

And the key thing to note here is that the fiber function just returns an object, and we're going to inspect that object in a second. But one of the methods that the object has is a run method. And so we can call that, and that will run our function which has been wrapped by this new fiber object. So when I Save that and reload the browser, we can step in and take a look at what a fiber looks like.

I've added a couple of debug statements here. And if we step through this, the first thing that's going to happen is the fiber gets created. And so what it's doing is it's wrapping this function called one and then it returns a new fiber object, which I'm storing in the My Fiber variable. And we can take a look at that in the right in the Scope Variable section.

And notice that the object has some prototype methods like Run, Reset, Throw Into, and so on. And what we're about to do is to call the Run method. And what that is going to do is to run our function, but it's going to run our function inside of this special construct called a fiber.

Before we step into it, I want you to take a look at the call stack over on the right and notice that it has all of these functions in it. That's going to change once we start running the fiber because each fiber is going to get its own call stack. So let's step through this code and this will drop us into our next debugger statement, which is in the first function there.

Now take a look over in the call stack and notice that it's brand new. We get a new call stack now, and there's only one function on it. It's the one that we're currently running from inside the fiber. And so each fiber gets its own call stack.

And just like before, we can call into these additional functions and we see the call stack on the right grow. So it starts off at one, and then we push two on and then push three on. And as the functions return, they get popped off of the call stack until finally the fiber run method returns and we're back in to the original call stack.

So notice that the call stack now has been replaced with what was there to begin with. And then we can just continue executing our code until we get down to the console log at the bottom. Now, what I'd like to do next is to pause execution of the fiber inside of one of these methods.

So inside of function two in the code on the left, what I can do is call a method that's on the global fiber object here called yield. And I'll put a debug statement there so that we can get there quickly. And what fiber.yield will do is pause execution of this fiber and store away its call stack, and then we can resume execution of the fiber later.

So what will happen here is we'll get dropped right back down at the bottom at this debug statement here. We can run our console.log. And then I can run the fiber again, which will resume it where we left off. And I can even pass it parameters. So let's pass something in like a parameter. And I can access that as the return value of yield.

So we'll just call this return value is equal to this. And then down here, I'll put another debugger statement so that we can see what happens. So we started off by creating the fiber and then call the Run method. And here we are in function two.

Notice the call stack has the one function and two function on it. And right at this point, we're going to call the Yield method of the global fiber object, an that's going to pause the currently running fiber. The reason that Yield is defined like this on the global fiber object is that there can only be one fiber running at a time, and so we can just have one global method called Yield.

Now, that will drop us right back to where we were before. Now, notice that we didn't continue executing down that function in two. We just got dropped right back into the main part of our code. And notice the call stack has been returned to the previous call stack, the one that we started with when we first started executing our program.

So what's happened here is right at this line in the code, the entire fiber has been stored away and saved to be resumed later. And that includes its call stack and the instruction pointer, or the place in the code where we left off. And in order to resume running this fiber, we have to call the Run method which we'll do down here.

So let's continue executing. And now this gives some additional code the ability to run. So right now, we are running more code and we're printing out, OK, I can run now. And at this point, we can resume running the fiber by just calling the Run method again. So if I press play, that's going to drop me right back to where we left off.

And the two things to notice that the call stack is returned to exactly what it was before and this return value, which we got from the returned value of calling this Yield function, is exactly what we passed into it down here. So notice that the return value is equal to pass in a parameter.

And then we can continue executing this function as usual until finally the call stack empties and this fiber finishes running. So fibers allow us to wrap a function, run that function, and then pause it during its execution at any point, and then resume it later whenever we want to.

In the next video, I'll show you how we can use fibers to get rid of the callback hell problem, but without losing the benefit of concurrent I/O that we get from using Node.js.