The way Meteor handles an app’s data is one of the framework’s greatest assets, but also one of the hardest things to wrap your head around when you’re just getting started.

This has led to a lot of misunderstandings, such as the belief that Meteor is insecure, or that Meteor apps can’t deal with large amount of data.

So today, I’d like to take a moment to clear up a few of these misconceptions by explaining Meteor publications and subscriptions.

The Olden Days

But first, let’s take a look back at the good old days of 2011 when Meteor wasn’t yet around. Let’s say you’re building a simple Rails app. When a user hits your site, the client (i.e. your browser) sends a request to your app, which is living on the server.

The app’s first job is to figure out what data the user needs to see. This could be page 12 of search results, Mary’s user profile information, Bob’s 20 latest tweets, and so on.

You can basically think of it as a bookstore clerk browsing through the aisles for the book you asked for.

Just imagine all this is one big database.

Once the right data has been selected, the app’s second job is translating that data into nice, human-readable HTML (or JSON in the case of an API).

In the bookstore metaphor, that would be wrapping up the book you just bought and putting it in a nice bag. This is the “View” part of the famous Model-View-Controller model.

Finally, the app takes that HTML code and sends it over to the browser. The app’s job is done, and now that the whole thing is out of its virtual hands it can just kick back with a beer while waiting for the next request.

The Meteor Way

The key innovation of Meteor is that where a Rails app only lives on the server, a Meteor app also includes a client-side component that will run on the client (the browser).

This is like a store clerk who not only finds the right book for you, but also follows you home to read it to you at night (which I’ll admit does sound a bit creepy).

This architecture lets Meteor do many cool things, chief among them what Meteor calls database everywhere. Simply put, Meteor will take a subset of your database and copy it to the client.

Pushing a subset of the database to the client.

This has two big implications: first, instead of sending HTML code to the client, a Meteor app will send the actual, raw data and let the client deal with it (data on the wire).

Second, you’ll be able to access that data instantaneously without having to wait for a round-trip to the server. And what’s even cooler is that Meteor will also do all the heavy lifting of keeping the client and server data in sync (latency compensation).

Managing Data

Our app might contain tens of thousands of documents, some of which might even be private or sensitive. So we obviously shouldn’t just mirror our whole database on the client, for security and scalability reasons.

So how exactly do you tell Meteor which subset of data to send to the client?

At this point the book metaphor breaks down (unless you start involving quantum books that can exist in two places at once) so let’s forget about it and look at some pretty diagrams instead.

First, here’s your data sitting in a database. We’ll imagine we’re building some kind of forum, and that our documents are user-submitted posts:

All the posts contained in our database.

Publishing

Some of our posts have been flagged for abusive language, and although we want to keep them in our database, they should not be made available to users (i.e. sent to a client).

So our first task will be telling Meteor what data we do want to send to the client. We’ll tell Meteor we only want to publish unflagged posts:

Excluding flagged posts.

Here’s the corresponding code, which would reside on the server:

// on the server
Meteor.publish('posts', function() {
  return Posts.find({flagged: false}); 
});

This ensures there is no possible way that a client will be able to access a flagged post. Not a chance.

This is exactly how you’d make a Meteor app secure: just ensure you’re only publishing data you want the current client to have access to.

Being More Flexible

What if we did want admins to be able to browse flagged posts?

Easy enough. Assuming we’ve already defined an isAdmin() function that checks permissions based on a user ID, we can simply do something like this:

// on the server
Meteor.publish('posts', function() {
    if(isAdmin(this.userId)){
        return Posts.find();
    }else{
      return Posts.find({flagged: false});
    }
});

Subscribing

Even though we want to make any non-flagged post available to clients, we can’t just send thousands of posts at once. We need a way for clients to specify which subset of that data they need, and that’s exactly where subscriptions come in.

Any data you subscribe to will be mirrored on the client thanks to Minimongo, Meteor’s client-side implementation of MongoDB.

For example, let’s imagine we’re currently browsing the profile page of Bob Smith, and only want to display his posts.

Subscribing to Bob’s posts will mirror them on the client.

First, we would ammend our publication to take a parameter:

// on the server
Meteor.publish('posts', function(author) {
  return Posts.find({flagged: false, author: author});
});

And we would then define that parameter when we subscribe to that publication in our app’s client-side code:

// on the client
Meteor.subscribe('posts', 'bob-smith');

This is how you make a Meteor app scalable client-side: instead of subscribing to all available data, just pick and choose the parts that you currently need. This way, you’ll avoid overloading the browser’s memory no matter how big your server-side database is.

Autopublish

If you’ve already played with a Meteor app, you might be scratching your head wondering why it seemed to work out of the box without having to worry about all this publish and subscribe nonsense.

By default, brand new Meteor apps include the autopublish package, which publishes and subscribes to all your data automatically to speed up early prototyping.

Once you remove it (as you should for any production app), you’ll find that your publications and subscriptions will behave as described in this article.

Finding

Now Bob’s posts happen to be spread across multiple categories (for example: “news”, ”classifieds”, and ”questions”). Maybe we still want to load up all of Bob’s posts, but we only want to display those from the “classifieds" category. This is where “finding” comes in.

Selecting a subset of documents on the client.

Just like we did on the server, we’ll use the Posts.find() function to select a subset of our data (in this case, so that we can use it through a {{classifieds}} Handlebars helper):

// on the client
Template.posts.helpers({
    classifieds: function(){
        return Posts.find(author: 'bob-smith', category: 'classifieds');
    }
});

We can now take Bob’s classifieds posts and display them in one tab, and (for example) take his “questions” posts and show them in another.

Going Further

Now that you understand how to publish and subscribe to data, the big question is how to do it in practice.

Should you just subscribe to as much data as you can when the user first connects? Or else just subscribe to what you need right now to speed up initial load, at the cost of introducing subsequent loading times within the app?

Exploring advanced pub/sub patterns in Discover Meteor .

There’s no one-size-fits-all answer. But if you’d like some help figuring things out, I’d recommend checking out our book. It includes a section on using Iron Router to manage subscriptions, as well as two chapters on different publication/subscription patterns.

In the meantime, if you have any questions about all this let us know in the comments!