• Meteor
  • Tutorial

The Reactive WaitList Data Structure

In this video, I'd like to show you a special reactive data structure that we use in Iron Router-- it's called a waitlist. I'm going to show you why we built this, and how you can use it directly. And then I'll show you a little bit about how it works.

In my sample application, I've added Iron Router to the project, and configured it not to auto start. So we're going to use Iron Router to get access to the waitlist, but we don't want to actually use the router itself. In the server block, I have two published functions-- one called 1, and the other one called 2-- and all they're going to do is to send their respective ready messages. So One will send a ready message in about three seconds, and then Two will send its in about five seconds, when we subscribe from the client.

Now what I want to do is in the client, subscribe to these publish functions, and then somehow wait until both of them are ready. To do that, I'll use the normal Meteor subscribe function, and we'll subscribe to one. But in Frontier, I'll assign this to a handle-- we'll just call it H1. So that will be the return value from calling subscribe. And then I'll subscribe to Two, and assign that to H2.

So now we have two subscription handles. And what I want to do is create a computation. And inside this computation, we'll see whether or not the first subscription is ready, and the second subscription is ready. And if they are both ready, then we'll print to the console, ready. And if not, we'll just print out that we're not ready yet.

In the console, first it says it's not ready. And then notice that we print out not ready again, and then the third time, we print out that we're ready. Let's go through this to make sure that it's making sense.

First, we subscribe to the two published functions up here, getting our handles, H1 and H2. And you can see those messages being sent up to the server, here. Then inside of our computation, we call the ready method of both handles, which will return true if we have the ready message, and will return false otherwise. But these functions are reactive, and so if the state of this changes, the entire function will be re-run.

Over in the right in the DDP message console, we can take a look at when we get these ready messages. And here's the subscription IDs, and so we'll look for those corresponding ready messages up here. And here's the first one, and then we get the second one a couple seconds later.

So what's happening here is first, we get the ready message for the first subscription, H1. And that causes H1 ready to go from false to true. And then this entire thing runs again.

And then a couple seconds later-- let me clear this up so we can see-- a couple seconds later we get the second ready message. And when that happens, H2'x ready state changes from false to true. And that causes this entire function to re-run again. And so we get these two not ready messages printed, and then finally, when both of these things are true-- when both of our subscriptions are ready-- we print out not ready.

So what I would really like is some kind of data structure where I can add these to a list. Kind of like an array, where I can just add H1 and H2 and then just say, if the list is ready, print out this ready. Otherwise, print out not ready.

And the second thing I'd like is I only want this function to re-run if the overall state of the list changes, not every single time we get a single ready message. We can use the waitlist to accomplish this.

First we need to do a little bit of a secret to get the waitlist out of the router package, because it's a privately scoped variable. So if I, in the console, type package, and then Iron Router, we can take a look at all of the different global symbols, or symbols that are available from this package-- and you can see that one of them is this waitlist object down at the bottom. So to get access to it, we can simply reference it directly from the package. Maybe someday we'll open this up so that you can just get to it directly. But for now, we can type package iron router, and then waitlist.

And then I'm going to create a new variable called List, and it'll be a new waitlist. And when I press Save and hot code push kicks in, we should have access to our list in the console.

OK, we get a little bit-- we get an error here, and that's because I didn't clean up this Auto Run. So let me do that now and save again.

And when this refreshes in the browser, let's play with the waitlist in the JavaScript console. I'll type list, and we can see what this instance of a waitlist looks like. If I open up its prototype, we can see that it has three methods that might be interesting to us.

One is the ready method, another is the wait method itself, and then we have a stop function if we want to stop everything in the list.

Back in our code, inside of the Auto Run function, let's add these two handles to our waitlist. We'll do that by calling the wait function. Now the wait function, itself, takes a function as a parameter, and that function should return a true or a false value reactively. In this case, the first one will just return the result of calling H1 ready. And we'll do the same thing again-- we'll pass a function, and it'll be true if the second handle is ready.

And then we'll say, if the entire list is ready, once again, we'll print out that we're ready. And if not, we'll print out that we're not.

Notice this time, we only print out not ready once instead of twice. And that's because one of the jobs of the waitlist is to only invalidate or only re-run its outer function if the overall state of the list changes from ready to not ready, or not ready to ready. So it saves some re-running of this function.

So we print out not ready, and then when the overall state of the list is ready, we say that we're ready.

Now, it turns out, the waitlist can work fine with just regular reactive variables, as well. We don't have to use subscriptions. So let's get rid of these subscriptions, and instead just create two new variables.

We'll use a new variable that comes with Blaze called Reactive Var. And as it sounds, it's just a reactive variable that can take a value. So we'll say that its initial value is false, and let's call the new keyword here, so that we actually create a new one. And then I'll get rid of the subscribe calls up here.

Now, instead of calling the Ready function here, I'm just going to call get. And then when I save this, and the code refreshers over on the right, we'll start off in a not ready state. And what I'm going to do is go over to the watch expressions and add one for a list, so we can take a look at what the data structure does as we change the ready state of these reactive variables.

So notice here that we have this not ready count, and that's set to two. And that's because we're waiting on two items here. We've called the Wait function twice, and that's why we have two items. And both of these items are currently not ready-- they're both returning false.

Now let's come down into the console and set one of these items to ready. We'll say that H1 is now true. And then I'll refresh the waitlist. And notice now the not ready count has been decremented to one. So now we only have one item in the list that isn't ready.

And at this point, if I were to call the Ready function of our list, it should still not be ready, because there's still one item left in there that's not ready. So now let's take H2 and set that to true. And now all items in the list are ready, and you see that our function actually re-ran printing out ready here.

But let's refresh this, and clean up the pen a little bit, and you can see that the not ready count is now zero. So all items in the list are ready. And that's when the waitlist re-runs it's computation function, and we print out that we're ready.

You can find the waitlist data structure source code in the Iron Router project, in the Lib Client directory, under the wait underscore list file. And what I suggest you do is to go into that file and take a look at the wait implementation. And you can see that right here.

So there's quite a bit of machinery going on here to make this work. If you'd like me to cover this algorithm in more detail, let me know. But in this video, we looked at why a waitlist data structure is necessary, and why we built it. And a little bit on how to use it-- both with subscriptions and custom functions.