User-friendly live collections in Ember.js

What is a live collection?

Demo

You need to display a fairly large list of data, say for example a list of tweets returned from a Twitter search. There will be new results every few seconds; do you really want your users to refresh your site every time they think there's new results? Not only would that be a bad user experience, it could wreak havoc on your server resources.

Instead let's use the magic of Javascript to rerun the Twitter search for the user on a regular basis. Using Ember.js we can search over and over again and watch the UI update every time we get new results.

In this article we'll see how we can leverage the strengths of Ember.js bindings to detect when new search results are available and display them without providing a jarring experience for the user.

User-friendly?

Like any other cool idea, this one has already been done by Thomas Reynolds. In his blog post, Thomas walks through building a Twitter search that updates every 2 seconds. As new tweets come in, they get added to the end of the list.

This is a great example, but I think we can improve the user experience here. First off, I'd like to see the most recent tweets at the top at all times. Of course, that means when new tweets arrive, the list would push down to make room for the new ones. This could lead to a pretty bad user experience if I search for a popular term.

Let's take this example a little further and improve the UI. Instead of displaying new tweets as they come in, we'll queue them up and display a prompt letting the user know that they have new results waiting for them. Then, we can wait to load them in until the user has indicated they are ready to view them (just like twitter.com does today).

Searching for Tweets

Let's assume we have a model, called App.Tweet that can search for tweets containing given keywords and hydrate a collection of App.Tweet instances with the properties of each tweet.

First, we perform a search for the hashtag "#flickr" in our route.

App.ApplicationRoute = Ember.Route.extend({
model: function(){
return App.Tweet.find("#flickr");
}
});

We'll need a controller to keep the collection of tweets, sorted by most recent first, for the view.

App.ApplicationController = Ember.ArrayController.extend({
sortProperties: ["created_at"],
sortAscending: false
});

It would be nice to format the tweets' timestamp similarly to Twitter (15 minutes ago, 1 hour ago, etc). Let's decorate the tweet model with an itemController and use moment.js to format the date.

App.ApplicationController.reopen({
itemController: 'tweet'
});

App.TweetController = Ember.ObjectController.extend({
formattedTimestamp: function(){
return moment(this.get("created_at")).fromNow();
}.property("last_updated_at")
});

Finally, we'll need a template to display the tweets. I'm using an extremely basic list to show the username, content and timestamp.

  <script type="text/x-handlebars">
<h1>Items</h1>
{{#if searchTerm}}
<p>Searched for {{searchTerm}}</p>
{{/if}}
<ul>
{{#each tweet in controller}}
<li>
<h3>{{tweet.from_user}}</h3>
<em>{{tweet.formattedTimestamp}}</em>
<p>{{tweet.text}}</p>
<small><em>({{tweet.id}})</em></small>
</li>
{{/each}}
</ul>
</script>

Live-updating the Results

Now that we have some search results displaying on the page, let's make them live-update. To do that, we're going to poll the Twitter search API at a set interval. Any time we get new results we'll call a refresh function on our model to add the new results to the array.

Polling like this is incredibly simple in Javascript, in its most basic form.

// Poll for new tweets every 5 seconds
setInterval(function(){
App.Tweet.refresh();
}, 5000);

Doing this with other APIs can get expensive, especially if you have a rate limit. Being able to disable polling can be extremely useful while you are debugging your app. Therefore, I like to wrap setInterval in a Javascript object that can handle starting and stopping the poll.

App.Pollster = {
start: function(){
this.timer = setInterval(this.onPoll.bind(this), 5000);
},
stop: function(){
clearInterval(this.timer);
},
onPoll: function(){
App.Tweet.refresh();
}
};

As users navigate around our app, we probably want our pollsters starting/stopping depending on what data the current route needs. In this case we have only one route so we can simply start the pollster when the route activates and stop it when the route deactivates.

App.ApplicationRoute.reopen({
activate: function(){
App.Pollster.start();
},
deactivate: function(){
App.Pollster.stop();
}
});

We now have a route that performs a Twitter search and populates an array controller with the search results. We have an item controller to decorate each tweet with a nicely formatted timestamp and then we display the tweets in a template. And of course, our results are automatically updated every 5 seconds.

But this isn't the user experience we want. Instead of pushing new tweets to the user, we want to queue them up and let the user choose when to load them in. For that, we'll have to use some of Ember's array observing magic.

Queueing Controller Content

The Power of Array Observers

To queue up new search results as they come in, we need to listen for array changes. New items get added to the model's array on almost every poll. When this happens, Ember triggers a chain of observers that work their way up through the controller and result in rerendering templates. One such observer is the array controller's content array (the same one that gets set in your route's setupController callback using controller.set("content", model);).

To detect when new items have been added, we're going to use the contentArrayDidChange callback on our controller. This will be called anytime changes are made to the structure of the content array (ie. items added, removed). This is an important distinction as there are other callbacks for different situations.

App.ApplicationController.reopen({
contentArrayDidChange: function(array, start, removeAmt, addAmt){
// do something to respond to this change
return this;
}
});

Let's talk about what's going on in the above code snippet. We declare a controller as an Ember.ArrayController. This actually makes our controller a child of Ember.ArrayProxy, which is a wrapper around native arrays with some special properties for observing changes to the array and its elements.

The callback contentArrayDidChange will be called whenever items are added to or removed from the instance's content. It gets the new array, the index at which the changes started, the number of elements removed and the number of elements added, respectively.

We can then use these parameters to create a separate array containing only new tweets; a "queue" if you will.

// snip...
contentArrayDidChange: function(array, start, removeAmt, addAmt){
var queued = [], added;
added = array.slice(start, start + addAmt);
added.forEach(function(item){
queued.push(item);
});
}
// ...snip

The above snippet is a naive implementation of queueing tweets for display. Since start is the index where the additions began, we can added addAmt to that to get the index where they end. Using Array#slice we'll get a new array containing only the added tweets. Then we can push each one onto our new array called queued.

Of course, this assumes that we'll add new tweets with every change. It's possible, however unlikely in this case, that our content array will change without any tweets being added. In that case, our final version will make sure we have added items.

The other problem with this implementation is that we're not saving the queue between changes. We'll need to add a property to our controller to hold onto the queue results until we're ready to display them.

App.ApplicationController.reopen({
queuedContent: function(){
return Ember.A();
}.property(),
contentArrayDidChange: function(array, start, removeAmt, addAmt){
var queued = this.get("queuedContent"), added;
if(start >= 0 && addAmt > 0){
added = array.slice(start, start + addAmt);
added.forEach(function(item){
if(!queued.contains(item)){
queued.pushObject(item);
}
});
}
return this._super.apply(this, arguments);
}
});

Filtering Array Content

Now we can call get("queuedContent") on our controller any time we want to get at the queued tweets! Of course, this seems pretty much useless, since our goal is to hide new tweets, not display them. Ah, but now that we know which tweets are queued, we can use that to filter them out of the tweets we want to display. How about we create one more property for that?

App.ApplicationController.reopen({
displayableContent: function(){
var queued = this.get("_queued");
return this.reject(function(obj){
var candidate = queued.findProperty("id", obj.get("id"));
return (typeof candidate !== "undefined" && candidate !== null);
});
}.property("queuedContent.[]"),
});

To get the tweets that we want to display, we're filtering down to only those that are not in the queue. Or, to say it as this function is written, we're rejecting any tweets that are in the queue.

Since this computed property observes queuedContent.[], any time items are added to, or removed from queuedContent, this property will update (if we just observed queuedContent without the [] it would only respond to a call to this.set("queuedContent", ...).

Let's look at what we have so far. We can search for tweets and poll for updates to the search results. We have a template that displays the search results and updates automatically as new ones come in. But they are getting pushed to user immediately, which is not what we want. Let's update our template to use our new displayableContent property.

{{#each tweet in displayableContent}}
<li>
<h3>{{tweet.from_user}}</h3>
<em>{{tweet.formattedTimestamp}}</em>
<p>{{tweet.text}}</p>
<small><em>({{tweet.id}})</em></small>
</li>
{{/each}}

So now we're hiding new tweets, but how do we notify the user as they come in? Let's add a couple properties to our controller to tell us if there are any queued tweets and how many.

App.ApplicationController.reopen({
queueCount: function(){
return this.get("queuedContent.length");
}.property("queuedContent.length"),
hasNewTweets: function(){
return this.get("queueCount") > 0;
}.property("queueCount"),
});

More Observer Goodness

When the user is ready to load in all the queued up tweets, we'll need a way to update the UI. Typically we'd have to iterate over our queue and manually insert each item into the DOM. But this is Ember.js. Thanks to the observer chain we have setup, updating the UI is as simple as emptying out our queue.

App.ApplicationController.reopen({
unqueueAll: function(){
this.get("queuedContent").clear();
}
});

What!? How could it possibly be that easy? Since our displayableContent computed property observes queuedContent.[], when we clear the array it will cause displayableContent to update. Since queuedContent is now empty (or, put simply, we have no more queued tweets), displayableContent is now a collection of all the tweets. Our template has an observer (thanks to Ember's {{#each}} helper) on displayableContent and will update whenever it changes.

Now let's update our template to show the number of tweets that are queued. We'll also add a button to load them in using the unqueueAll action.

{{#if hasNewTweets}}
You have {{queueLength}} new tweets.
<button {{action unqueueAll}}>Load</button>
{{/if}}

Conclusion

Well, there you have it. A user-friendly, live updating collection written in Ember.js. If you found any part of this article confusing, I highly recommend reading it a couple times and looking up any methods you don't know in the Ember API docs or by searching the source code (for example, the contentArrayDidChange method is not in the API docs but is well-commented and tested).

Ember.js can be quite overwhelming at first, but as you start to grasp its heavy use of the observer pattern, it becomes a lot less magic. Once you start letting your framework handle the heavy-lifting of model-observing and DOM updating, you can focus more on building the best UI for your users.

As always, feel free to hit me up on Twitter or App.net with any questions or feedback.

Published on by Joe Fiorini.