igvita.com

Building a Modern Web Stack for the Real-time Web

2012-01-18T00:00:00-08:00

The web is evolving. After a few years of iteration the WebSockets spec is finally here (RFC 6455), and as of late 2011 both Chrome and Firefox are SPDY capable. These additions are much more than just "enhancing AJAX", as we now have true real-time communication in the browser: stream multiplexing, flow control, framing, and significant latency and performance improvements. Now, we just need to drag our "back office" - our web frontends, app servers, and everything in between into this century to enable us to take advantage of these new capabilities.

We're optimized for "Yesterday's Web"

Modern backend architecture should allow you to terminate the user connection as close to the user as possible to minimize latency - this is your load balancer or web server occupying ports 80 and 443 (SSL). From there, the request is routed on the internal network from the frontend to the backend service, which will generate the response. Unfortunately, the current state of our "back office" routing is not only outdated, but often is also the limiting factor in our adoption of these real-time protocols.

WebSockets and SPDY are both multiplexed protocols which are optimized to carry multiple, interleaved streams of data over the same TCP pipe. Unfortunately, popular choices such as Apache and Nginx have no understanding of this and at best degrade to dumb "TCP proxies". Even worse, since they do not understand multiplexing, stream flow-control and priority handling goes out the door as well. Finally, both WebSockets and SPDY communicate in framed messages, not in TCP streams, which need to be re-parsed at each stage.

Put all of this together and you quickly realize why your own back office web stack, and even the popular platforms such as Heroku and Google's App Engine are unable to provide WebSockets or SPDY support: our services are fronted by servers and software which was designed for yesterday's web.

Architecture for the "Real-time Web"

HTTP is not going away anytime soon, and we will have to support both the old and the new protocols for some time to come. One attempt at this has been the SPDY > HTTP proxy, which converts a multiplexed stream into a series of old-fashioned HTTP requests. This works, and it allows us to reuse our old infrastructure, but this is exactly backwards from what we need to be doing!

Instead of converting an optimized, multiplexed stream into a series of internal HTTP dispatches, we should be asking for HTTP > SPDY infrastructure, which would allow us to move beyond our outmoded architectures. In 2012, we should demand our internal infrastructure to offer the following:

Request and Response streaming should be the default
Connections to backend servers should be persistent
Communication with backend servers should be message-oriented
Communication between clients and backends should be bi-directional

Make SPDY the default, embrace dynamic topologies

The first step towards these goals is to recognize that translating SPDY to HTTP is a convenient path in the short term, but exactly the wrong path in the long term. SPDY offers multiplexing, flow control, optimized compression, and framing. We should embrace it and make it the default on the backend. Once we have a multiplexed, message-oriented protocol on the backend, we can also finally stop reparsing the same TCP stream on every server. Writing HTTP parsers in 2012 is neither fun nor an interesting problem.

Finally, this architecture should not require a dedicated OPS team, or a custom software platform to maintain. Modern web applications are rarely powered by a single host and require dynamic (re)configuration and management. Services such as Heroku, CloudFoundry, and GAE have built their own "routing fabrics" to handle these problems. Instead, we need to design architectures where the frontends and the backends are decoupled by default and require minimal intervention and maintenance.

Adopt a modern Session Layer

Building dynamic network typologies is not for the faint of heart, especially once we add the additional requirements for message-oriented communication, multiplexed streams and a grab bag of performance constraints. Thankfully, libraries such as ØMQ offer all of the above and more, all wrapped behind a simple and an intuitive API. Let the frontend parse and emit SPDY frames, and then route them internally as ØMQ messages to any number of subscribers.

Mongrel2 was one of the first web servers to explore this type of architecture with ØMQ, which allowed it to sidestep the entire problem of backend configuration, as well as enable a number of interesting worker topology patterns. There is still room for improvement, but it is a much needed step in the right direction. As a concrete example, let's consider a sample workflow with SPDY and ØMQ:

An HTTP (or SPDY) request arrives to the frontend
Frontend parses the request and generates SYN_STREAM, HEADERS and DATA SPDY frames
The messages are delivered into a PUSH ØMQ socket (ala Mongrel2)
Backend subscribers use a PULL socket to process the SPDY stream
Backend subscriber streams a response back to the frontend

The communication is done over a persistent channel with message-oriented semantics, the frontend and the backends are completely decoupled, and we can finally stop punching "TCP holes" in our networks to support the modern web.

Supporting HTTP 2.0 in the back office

The new protocols are here, but the supporting "back office" architecture requires a serious update: SSL is becoming the default, streaming is no longer an option, and long-lived persistent connections are in. SPDY is gaining momentum, and I have no doubts that in the not so distant future it will be an IETF approved protocol. Similarly, ØMQ is not the only alternative for internal routing, but it is definitely one that has been gaining momentum.

Fast HTTP parsing and routing is simply not enough to support the modern web use cases. Likewise, punching "TCP holes" in our infrastructure is not a viable long-term solution - in 2012 we should be asking for more. Yes, I'm looking at you Varnish, Nginx, Apache and friends.

Google Web Toolkit: GWT, Closure, NaCl, Dart & JS

2012-01-05T00:00:00-08:00

As web developers, one thing we're not short on is the number of options when it comes to web frameworks, languages and runtimes. Google alone is now officially supporting at least five different platforms: JavaScript, GWT, Closure, and the two recent additions in NaCl and Dart.

On one hand, it is easy to look at this list and wonder why all the dissonance and competition. After all, all of these teams are within the same company, can't they just agree to work together? However, if you look at the full scope of these projects, a slightly more interesting perspective emerges: it is not about trying to find the "one true way" to build a modern web application, instead, intentionally or not, Google is funding and providing the tools for every take at the problem.

The "Google Web Toolkit"

Regardless of the technology, the end result, at the center of the diagram, is the plain old HTML page, which is executed by the browser. Both of these technologies is something that Google has put a lot of investment in: heavy involvement with HTML5, and of course the Chrome browser itself.

However, most web apps today are not just static HTML, to add interactivity we have historically relied on JavaScript, which is where the Chrome V8 runtime comes in with its focus on speed. "Fast is a feature", especially when you have apps like GMail which consist of 443,000+ lines of handrolled JavaScript!

Developing in raw JavaScript can be fun, but the lack of tooling can be a problem at scale. Closure is one attempt at solving this problem by introducing a dedicated linter/compiler, which also adds features like namespaces, type checking, data hiding, a "standard library", and a number of other extensions. In effect, you write Closure flavored JavaScript, you pass it through the compiler, and the output is optimized and minified JavaScript. The Google+ team has professed their love for Closure on numerous occasions, and they are just one of the many teams within Google using it in production.

Structured Web Programming with GWT & Dart

But, you say, I love the Eclipse toolchain, Java and all the infrastructure that comes with it - if only we could use that to build our web applications! That's Google Web Toolkit (GWT). The GWT SDK allows you to write your entire application in Java and then compile the source into optimized JavaScript. In fact, that's exactly how Google AdSense is built. And for a more "entertaining" example, add a framework like PlayN on top of GWT, and you get AngryBirds in the browser!

However, not everyone is a fan of Java. What if we had a language optimized for "structured web programming"? Say hello to Dart. The language is still in technology preview, but the goal is to take a comprehensive approach at the problem: structured, familiar language, common libraries, modularity, and built-in tool support. Boot up your Dart editor (Eclipse based), develop and debug your app, and the Dart compiler will emit optimized JavaScript when you ship the product.

Life beyond JavaScript

Wherever there is a browser, there is a JavaScript engine. This makes JavaScript an incredibly powerful target. However, why should we pin the future of the web on JavaScript alone?

Dart, in addition to compiling to JavaScript also offers the "Dart VM", which will execute the code directly on your local workstation or server. In fact, you can even download and build Dartium, which is a Chromium build with a built-in Dart VM - no JavaScript required!

<html>
  <body>
    <script type='application/dart'>
      void main() {
        HTMLElement element = document.getElementById('message');
        element.innerHTML = 'Hello from Dart';
      }
    </script>
    <div id='message'></div>
  </body>
</html>

But if we can have Dart running natively in browser, what about all the other languages? That's Native Client (NaCl), which allows you to run native compiled code directly in your browser! Grab your C/C++ source, compile it with the NaCl toolchain, and you can have your physics engine, game, or any other CPU intensive app run directly in the browser.

Additionally, there is ongoing work on Portable NaCl (PNaCl), which will accept any LLVM bytecode and run it in NaCL - which, in turn, means that we can run Python, Ruby, Lua, and dozens of other languages right in the browser. For the curious, see my RubyConf 2011 talk on this subject below (slides):

The Google Coopetition

It is easy to look at all the alternatives and wonder, what does Dart mean for GWT, or Closure, how does NaCl affect the picture, and what about the good old JavaScript itself? What's the "one true way" to build a modern web app in 2012? The short answer is: wrong question, the answer depends on your context.

Google itself has many high profile products powered by each option above, and each has its strengths and weaknesses, as well as an audience for which it is the "right fit". Instead of trying to bet on any single solution Google is systematically investing into each and every alternative route - call it cooperative competition.

Don't "Push" Your Pull Requests

2011-12-19T00:00:00-08:00

You've run into a problem, or you have an itch to scratch, so you fire up the editor and dive right in. On a good occasion the patch is simple, so you pat yourself on the back, fire off a pull request and sit back and watch - even though it is a simple patch, past experience shows that often you will have to incorporate some feedback before it will be merged upstream.

Then, every once in a while, you stumble on that gnarly problem - the one that looked simple on the surface - and you spend the better part of your day on it, or more, and the further you get in, the more committed you become to see it through. Finally, you emerge victorious with a 300+ line patch, a dozen modified files, and a feeling that you have just done the world a favor - time to fire off the pull request!

This is where it all breaks down. You expect gratitude for all the time, work, and effort you've put in, but the maintainer looks in horror at the patch as he tries to enumerate all the reasons why he can't accept it. Half the time, he doesn't even know where to start, so he picks a nitpick and digs in - your contribution is going nowhere fast and neither side is to blame.

Don't "push" the pull request

Contributions are always welcome, but surprise patches are mostly just a burden. Yes, you are offering your help, but someone else will have to maintain your code over the long term - get their buy-in first, avoid surprises. Even worse, a localized change to address a specific problem will often miss the full implications to the project: other existing use cases, future roadmap plans, or overall architectural decisions. A good idea can be implemented inappropriately for the specific project; it can be invalidated by another effort that you may not even be aware of; the timing may be wrong, and a dozen other reasons can conspire against you.

Those responsible for maintaining the code have the best perspective, so engage them early. That is not to say they are always right, and you may have to make your case regardless, but this is a discussion better had before you write the code, not after. This is one of the most frustrating paradoxes in open-source: all too often it is your most engaged fans that fall into this trap, spending their nights and weekends working on a massive contribution, only to be disappointed when the maintainer flat out rejects their work.

Code reviews are hard: start early

Next time, before you dive into the editor, open a discussion, outline your problem and solution and offer to do the work. It shouldn't take more than a paragraph to explain the goal - if you need an essay, that's a sign already that you need to break it up, or the problem is not yet well defined. If the project or the change is large, pick a specific reviewer and go over the design. Ask for help and input early, and magically, you will find your code getting "pulled" (merged) on schedule.

Code reviews are hard. In fact, as it turns out, the effectiveness of a code-review drops dramatically after 200-400 lines of code, and the suggested inspection rate is at most 300-500 lines an hour. Keep this in mind, break down large commits into small, manageable chunks. As a rule of thumb, shoot for half an hour of review, or less than 200 lines with good and clear (not clever) code. If there are multiple commits, then provide a roadmap, and get feedback on each piece as it comes together.

Good work gets "pulled" not "pushed"

There is nothing more frustrating than to read the threads, or even worse, be on the receiving end of this experience gone wrong. The contributor is wounded because they are not recognized, and the maintainer is caught in a dilemma: they want to build and foster their community, but they have to dig-in against their biggest fans. There is no winner here and neither side is to blame.

Next time, before you dive into the editor, open a discussion, outline your problem and solution, and offer to do the work - don't "push" the pull request, and don't work in the dark.

Web-VPN: Secure Proxies with SPDY & Chrome

2011-12-01T00:00:00-08:00

Amazon's recent launch of their new Kindle Fire device delivered an interesting surprise: their Silk browser routes all requests via an Amazon powered SPDY proxy! In principle, this is not new; Opera, Blackberry and many other mobile providers have been using similar approaches to "optimize" the browsing experience for some time. However, Silk's announcement immediately generated a slew of security outcries: doesn't this mean that Amazon is now effectively a man in the middle (MITM) for all of our sessions, including SSL? Is this all just a big data mining ploy, or worse?

Turns out, the answer is: none of the above. Amazon does not route SSL traffic through their SPDY proxies, instead all HTTPS requests are routed directly to the destination. However, this incident did surface an interesting underlying question: is it, in fact, possible for a web browser to securely route SSL sessions via a web-proxy?

Tunneling SSL over HTTP

First, it is worth recalling that HTTP does allow us to tunnel SSL connections, end-to-end, via an HTTP proxy. This is a little known protocol feature, but an important one for our purposes:

To proxy an HTTPS session the client connects to the proxy by sending a special CONNECT HTTP method, which also includes just the host and the port of the destination. The proxy authenticates the request, completes the TCP handshake with the specified destination and returns a 200 Connection Established response, after which, it simply becomes a dumb, two-way router. At this point, the client and the server negotiate the SSL session and start to exchange encrypted data directly through the proxy. In other words, the proxy only sees encrypted data, and the only thing it knows are the hostnames and the ports of both parties.

This is great, but there is still one problem: the original CONNECT handshake (in green), alongside the host and the port of the destination is sent in clear text. Even worse, the proxy authentication (if any) is also running over the same insecure channel. Unfortunately, browsers today do not support secure HTTP proxies! Hence the reason why we've historically resorted to heavyweight SSL VPN's, SOCKS tunnels, and the like.

Chrome, SPDY and Secure Web Proxies

Except, as it turns out, what we said above is not entirely true: earlier this year, Google Chrome added support for secure web proxies! Since all SPDY sessions run over SSL, a SPDY proxy, by definition, would need to be SSL capable. Hence, the Chrome team went ahead and added secure web proxy support - nice.

Browsers today do not support secure proxies. Although proxies can tunnel SSL, connectivity to the proxy itself is only over HTTP. To support SPDY, we need to modify Chromium to support a SSL-based proxy.

Web-VPN & Chrome

Enabling SSL for the first hop to the proxy seems like a minor change - and it did go largely unnoticed - but it opens up some really interesting use cases! For one, the authentication can now be handled at a certificate level, which can be provisioned, revoked, and verified securely. In other words, your Chrome browser is now also a "Web-VPN client" - except without the annoying installation, configuration, and maintenance!

How exactly would Web-VPN work? Simply deploy a secure HTTP proxy as a gateway to any private network, point your Chrome browser at it, and voila, you will be able to resolve any internal hostname or service just as if you were running a VPN client. The first hop to the proxy runs over SSL, and all subsequent tunnels as well.

SPDY: 2012 is the year!

One of the primary motivations for SPDY is to reduce the overall page loading time. However, while speed is important, the security aspects are arguably even more critical - support for secure web proxies is one of many great examples.

Interestingly, Chrome is on track to take the #2 browser spot away from Firefox before the end of the year, and at the same time, Firefox 11 will arrive on December 20th. So what? Well, one of the less often mentioned features of Firefox 11 is the built-in SPDY support! Combined, between Chrome and Firefox, this means that over 50% of all internet sessions will be SPDY capable. Mark 2012 as the beginning of the end for HTTP.

Faster Web vs. TCP Slow-Start

2011-10-20T00:00:00-07:00

Ever obsess about trying to find the "best and fastest" internet provider? Having recently gone through the marketing vortex of comparing a dozen different plans, I was then reminded of a simple fact: the primary metric (bandwidth) used by the industry is actually highly misleading. It turns out, for most web-browsing use cases, an internet connection over several Mbps offers but a tiny improvement in performance.

One of the main culprits is "TCP slow-start", and yes that is a feature, not a bug. To understand why, we have to look inside of the TCP stack itself, and in doing so we can also learn a few interesting tips for how to build faster web-services in general.

TCP Slow-start

TCP offers a dozen different built-in features, but the two we are interested in are congestion control and congestion avoidance. TCP slow-start specifically is an implementation of congestion control within the TCP layer:

Slow start is used in conjunction with other algorithms to avoid sending more data than the network is capable of transmitting, that is, to avoid causing network congestion.

The high-level flow is simple: the client sends a SYN packet which advertises its maximum buffer size (rwnd - receive window), the sender replies by sending several packets back (cwnd - congestion window) and then each time it receives an ACK from the client, it doubles the number of packets that can be "on the wire" while unacknowledged.

This phase is also known as the "exponential growth" phase of the TCP connection. OSI school has an an excellent animation illustrating this phase: scroll to the bottom and hit play.So, why do we care? Well, no matter what the size of your pipe, every TCP connection goes through this phase, which also means that more often than not, the utilized bandwidth is effectively limited by the settings of the sender's and receiver's buffer sizes.

HTTP and TCP slow start

Of course, a higher bandwidth connection will help when you are streaming a large file, or running a vanity speed test. The problem is, HTTP traffic tends to make use of short and bursty connections - in these cases we often never even reach the full capacity of our pipes! Research done at Google shows that an increase from 5Mbps to 10Mbps results in a disappointing 5% improvement in page load times.

Or put slightly differently, a 10Mbps connection, on average uses only 16% of its capacity. Yikes! As it turns out, if we want faster internet, we should focus on cutting down the round-trip time between the client and server, not necessarily just investing in bigger pipes.

The story of CWND

If TCP slow start is, well, slow, then couldn't we just make it faster? Turns out, until very recently the Linux TCP stack itself was hardcoded to start with the congestion window (cwnd) of just 3 or 4 packets, which amounts to about 4kb (~1360 bytes per packet). Combine that with the unfortunately frequent pathological case of fetching a single resource per connection, and you have managed to severely limit your performance.

As of kernel version 2.6.33, following a protracted discussion and a number of IETF recommendations, the initial cwnd value has been reset to 10 packets, which in itself is a huge step forward. Only one problem, guess what kernel versions most servers run today? Yes, perhaps it is time to upgrade your servers.As a practical tip, if you have been thinking about enabling SPDY for your web-service, then running on anything but some of the latest kernels won't actually give you any performance improvements! A tiny change in the TCP stack, but a big difference overall.

So what can I do about it?

TCP slow start is a feature, not a bug, and it does carry some interesting and important implications. As developers we often overlook the round trip time from the client to the server, but if we are truly interested in building a faster web, then this is a good time to investigate your options: re-use your TCP connections when talking to web-services, build web-services that support HTTP keep-alive and pipelining, and do think about end-to-end latency. Oh, and don't waste your money on that 10Mbps+ pipe, you probably don't need it.

Optimizing HTTP: Keep-alive and Pipelining

2011-10-04T00:00:00-07:00

The last major update to the HTTP spec dates back to 1999, at which time RFC 2616 standardized HTTP 1.1 and introduced the much needed keep-alive and pipelining support. Whereas HTTP 1.0 required strict "single request per connection" model, HTTP 1.1 reversed this behavior: by default, an HTTP 1.1 client and server keep the connection open, unless the client indicates otherwise (via Connection: close header).

Why bother? Setting up a TCP connection is very expensive! Even in an optimized case, a full one-way route between the client and server can take 10-50ms. Now multiply that three times to complete the TCP handshake, and we're already looking at a 150ms ceiling! Keep-alive allows us to reuse the same connection between different requests and amortize this cost.

The only problem is, more often than not, as developers we tend to forget this. Take a look at your own code, how often do you reuse an HTTP connection? Same problem is found in most API wrappers, and even standard HTTP libraries of most languages, which disable keepalive by default.

HTTP Pipelining

The good news is, keep-alive is supported by all modern browsers and mostly works out of the box. Unfortunately, support for pipelining is in a much worse off shape: no browsers support it officially, and few developers ever think about it. Which is unfortunate, because it can yield significant performance benefits!

While keep-alive helps us amortize the cost of creating a TCP connection, pipelining allows us to break the strict "send a request, wait for response" model. Instead, we can dispatch multiple requests, in parallel, over the same connection, without waiting for a response in serial fashion. This may seem like a minor optimization at first, but let's consider the following scenario: request 1 and request 2 are pipelined, request 1 takes 1.5s to render on the server, whereas request 2 takes 1s. What is the total runtime?

Of course, the answer depends on the amount of data sent back, but the lower bound is actually 1.5s! Because the requests are pipelined, both request 1 and request 2 can be processed by the server in parallel. Hence, request 2 completes before request 1, but is sent immediately after request 1 is complete. Fewer connections, faster response times - makes you wonder why nobody advertises that their API supports HTTP pipelining?

HTTP Keep-alive & Pipelining in Ruby

Unfortunately, many standard HTTP libraries revert to HTTP 1.0: one connection, one request. Ruby's own net/http uses a little known behavior where by default an "Connection: close" header is appended to each request, except when you're using the block form:

require 'net/http'

start = Time.now
Net::HTTP.start('127.0.0.1', 9000) do |http|
  r1 = http.get "/?delay=1.5"
  r2 = http.get "/?delay=1.0"

  p Time.now - start # => 2.5 - doh! keepalive, but no pipelining
end

With the example above we get the benefits of HTTP keep-alive, but unfortunately net/http offers no support for pipelining. To enable that, you'll have to use a net-http-pipeline, which is a standalone library:

require 'net/http/pipeline'

start = Time.now
Net::HTTP.start 'localhost', 9000 do |http|
  http.pipelining = true

  reqs = []
  reqs << Net::HTTP::Get.new('/?delay=1.5')
  reqs << Net::HTTP::Get.new('/?delay=1.0')

  http.pipeline reqs do |res|
    puts res.code
    puts res.body[0..60].inspect
  end

  p Time.now - start # => 1.5 - keep-alive + pipelining!
end

EM-HTTP & Goliath: Keep-alive + Pipelining

While pipelining is disabled in most browsers, due to many issues related to proxies and caches, it is nonetheless a useful optimization for your own, or for talking to your partner API's. The good news is, Apache, Nginx, HAproxy and others support it, but the problem is that most app servers, even the ones which claim to be "HTTP 1.1", usually don't.

True keep-alive and pipelining support is one of the reasons we built both em-http-request and Goliath for our stack at PostRank. A simple example in action:

require 'goliath'

class Echo < Goliath::API
  use Goliath::Rack::Params
  use Goliath::Rack::Validation::RequiredParam, {:key => 'delay'}

  def response(env)
    EM::Synchrony.sleep params['delay']
    [200, {}, params['delay']]
  end
end

require 'em-http-request'

EM.run do
  conn = EM::HttpRequest.new('http://localhost:9000/')
  start = Time.now

  r1 = conn.get :query => {delay: 1.5}, :keepalive => true
  r2 = conn.get :query => {delay: 1.0}

  r2.callback do
    p Time.now - start # =>  1.5 - keep-alive + pipelining
    EM.stop
  end
end

Total runtime is, you guessed it, 1.5s. If your public or private API's are built on top of HTTP, then keep-alive and pipelining are features you should be leveraging wherever you can.

Optimizing HTTP: Interrogate your code!

While we love to spend time optimizing our algorithms, or making the databases faster, we often forget the basics: setting up TCP connections is expensive, and pipeling can lead to big wins. Do use reuse HTTP connections in your code? Does your app server support pipelining? The answers are usually "no, and I'm not sure", which is something we need to change!

Server-Sent Event Notifications with HTML5

2011-08-26T00:00:00-07:00

Server-Sent Events (SSE) have long been in the shadow of the much more often talked about WebSockets API. While WebSockets can be thought of as the "TCP for the web", then SSE is best described as the "HTML5 replacement for Comet)". Question is, do we really need both?

WebSockets allow bi-directional communication and to do so, they piggyback on the original HTTP connection and "upgrade" to the WebSocket protocol. By comparison, and as the name implies, SSE is strictly a server push protocol. Additionally, messages are encoded in UTF-8 and are newline delimited. In other words, the bet is that most applications primarily need to push data to the client, which means that we can simplify the API and the implementation - that is where SSE comes in. Still skeptical? I was. Let's take a look under the hood.

EventSource Client API

Unlike a WebSocket connection, SSE requires no additional HTTP handshakes or "protocol upgrades". For all intents and purposes, an SSE channel is simply a long-lived, streaming HTTP connection. This alone means that you can implement an SSE service on top of any streaming capable HTTP server.

On the browser side, the EventSource API is now supported by Chrome, Firefox 6+, Safari and Opera. To start receiving server notifications, we simply open a connection to an SSE URI and setup our callbacks:

var source = new EventSource('/my/sse_endpoint');

source.addEventListener('message', function(e) {
  console.log(e.data);
}, false);

source.addEventListener('user_login', function(e) {  
  console.log(e.data);
}, false);

source.addEventListener('open', function(e) {
  // Connection was opened.
}, false);

source.addEventListener('error', function(e) {
  if (e.eventPhase == EventSource.CLOSED) {
    // Connection was closed.
  }
}, false);

The API is very simple: create an EventSource connection, define your connection open, close, and message callbacks, and you are off to the races. However, the SSE API also provides a few more built-in features. In the example above we added a "user_login" listener - turns out, we can tag our messages! Likewise, we can provide message ID's, and the SSE source will even automatically reconnect for you if the connection is dropped.

Simple SSE Server with Goliath

The client API is nice and simple, but where the SSE spec really shines is in how simple it makes the server implementation. Whereas to power a WebSocket service you will likely need an entirely different backend, an SSE endpoint can be implemented by any streaming capable HTTP web server. As an example, let's create a Goliath powered SSE app and deploy it to Heroku:

require 'goliath'

class EventGenerator < Goliath::API
  def response(env)
    EM.add_periodic_timer(1) { env.stream_send("data:hello ##{rand(100)}\n\n") }
    EM.add_periodic_timer(3) do
      env.stream_send(["event:signup", "data:signup event ##{rand(100)}\n\n"].join("\n"))
    end

    streaming_response(200, {'Content-Type' => 'text/event-stream'})
  end
end

class SSE < Goliath::API
  use Rack::Static, :urls => ["/index.html"], :root => Goliath::Application.app_path("public")

  get "/events" do
    run EventGenerator.new
  end
end

Our Goliath server responds to requests to /events by returning a 200 OK response, and then starts emitting newline delimited messages every couple of seconds - we're streaming plaintext messages directly into the HTTP connection! Additionally, Goliath also serves a static index file, which opens an EventSource connection to our /events endpoint (see the full gist here). With that, we can add a Procfile and a Gemfile, and push it out to Heroku: a live SSE demo powered by Goliath.

SSE vs. WebSockets

Technically speaking, the WebSockets spec is a superset of SSE and it is easy to wonder why we need both. Having said that, it is also hard not to like the simplicity of the EventSource API - both on the client, and especially on the server. Chances are, many apps won't actually need the bi-directional capabilities of WebSockets, and hence can benefit a great deal from the simplified setup and deployment enabled by SSE. Definitely an option to keep in mind.

Rails 3 Asset Pipeline & Google Closure

2011-08-16T00:00:00-07:00

Rails 3.1.0 is on the horizon and the new Asset Pipeline is the king of the show. If this comes as news, then make sure to check out DHH's keynote from RailsConf, and also read through the excellent edge guide. This new release brings about some of the most developer visible changes in a long time: it both makes the previously "stashed away" assets such as Javascript and CSS first class citizens in our development workflow, and also pushes us into the bold new world of Coffeescript, Sass, and many other similar tools. It's about time!

Asset Pipeline & Preprocessing

Under the hood, there are two major components: Sprockets is responsible for all the asset preprocessing, and there is an additional middleware (mounted under /assets) which manages the actual toolchain. Hence, a request to /assets/application.js will hit the middleware, which will in turn invoke Sprockets, run the minifiers and compressors, and finally return the Javascript file. Good news is, this is all basically invisible.

Let's take a closer look at the default application.js:

// javascript comment
//= require jquery
//= require mylib/stuff.js
//= require_tree otherlib/

The goal is to enable Rails to serve a single Javascript file back to our visitors. To accomplish this, we need to tell Rails which files and directories should be included in the output. With that in mind, and as you may have guessed, the //= comment is, in fact, a special directive for Sprockets to include the specified file in the processing of the asset pipeline. Just specify the files, or entire directories and the rest is taken care of. CSS handling is no different.

Google Closure + Dependency Management

Turns out, Google's Closure library has a similar, but different, built-in dependency management system:

<script src="closure-library/closure/goog/base.js"></script>
<script>goog.require('goog.dom')</script>
<script>
  var newHeader = goog.dom.createDom('h1', {}, 'Hello world!');
</script>

Closure's DOM manipulation (goog.dom) is an independent set of libraries, which are loaded by Closure at runtime when we call goog.require. Now, the dynamic loading is convenient in development mode, but it would be both slow and costly in production! To address this, Closure also ships with a separate Python script to preprocess your Javascript to generate the single asset file:

$ closure-library/closure/bin/calcdeps.py -i hello.js -p closure-library/ -o script > hello-calc.js

In other words, the net effect is the same as that provided by the built-in Asset Pipeline, but it requires manual intervention and third party tools - we can do better.

Extending Rails 3 Asset Pipeline: Closure support

The great news is, Sprockets allows us to easily modify and add new directive processors into our pipeline: there is no reason why we should be limited to //= directives only! Closure's goog.require's are effectively the same mechanism, hence we can extract these calls, load the right files, and make Closure a first class citizen in Rails! To do that, we can define a simple Closure Preprocessor:

class ClosureDependenciesProcessor < Tilt::Template
  def evaluate(context, locals, &block)
    context.require_asset 'goog/base'

    data.lines.each do |line|
      # scan the JS file for goog.require calls
      if line.match('goog\.require\s*\(\s*[\'\"]([^\)]+)[\'\"]\s*\)')
        goog, mod, sub = $1.split(".")
        sub = mod if sub.nil?

        dep = [goog, mod, sub].compact.join("/").downcase
        context.require_asset(dep)
      end
    end

    data
  end
end

closure-sprockets - Sprockets processor for Google's Closure tools

With that in place, we can define a new Railtie to register the new Javascript preprocessor, package it as a gem, and we're in business! Add the gem to your Gemfile, download the closure library, and you are good to go.

Rails 3 API's

The final closure-sprockets gem is almost trivial and is even easier to setup within your own project. Having said that, this is only true because of the extensibility provided by both the Sprockets preprocessors and the Railtie integration - kudos to the Rails 3 team for putting together these great API's. Finally, thanks to Josh for hand-holding me through the process of learning about Sprockets and the new Asset Pipeline!

Protocol Buffers, Avro, Thrift & MessagePack

2011-08-01T00:00:00-07:00

Perhaps one of the first inescapable observations that a new Google developer (Noogler) makes once they dive into the code is that Protocol Buffers (PB) is the "language of data" at Google. Put simply, Protocol Buffers are used for serialization, RPC, and about everything in between.

Initially developed in early 2000's as an optimized server request/response protocol (hence the name), they have become the de-facto data persistence format and RPC protocol. Later, following a major (v2) rewrite in 2008, Protocol Buffers was open sourced by Google and now, through a number of third party extensions, can be used across dozens of languages - including Ruby, of course.

But, Protocol Buffers for everything? Well, it appears to work for Google, but more importantly I think this is a great example of where understanding the historical context in which each was developed is just as instrumental as comparing features and benchmarking speed.

Protocol Buffers vs. Thrift

Let's take a step back and compare Protocol Buffers to the "competitors", of which there are plenty. Between PB, Thrift, Avro and MessagePack, which is the best? Truth of the matter is, they are all very good and each has its own strong points. Hence, the answer is as much of a personal choice, as well as understanding of the historical context for each, and correctly identifying your own, individual requirements.

When Protocol Buffers was first being developed (early 2000's), the preferred language at Google was C++ (nowadays, Java is on par). Hence it should not be surprising that PB is strongly typed, has a separate schema file, and also requires a compilation step to output the language-specific boilerplate to read and serialize messages. To achieve this, Google defined their own language (IDL) for specifying the proto files, and limited PB's design scope to efficient serialization of common types and attributes found in Java, C++ and Python. Hence, PB was designed to be layered over an (existing) RPC mechanism.

By comparison, Thrift which was open sourced by Facebook in late 2007, looks and feels very similar to Protocol Buffers - in all likelihood, there was some design influence from PB there. However, unlike PB, Thrift makes RPC a first class citizen: Thrift compiler provides a variety of transport options (network, file, memory), and also tries to target many more languages.

Which is the "better" of the two? Both have been production tested at scale, so it really depends on your own situation. If you are primarily interested in the binary serialization, or if you already have an RPC mechanism then Protocol Buffers is a great place to start. Conversely, if you don't yet have an RPC mechanism and are looking for one, then Thrift may be a good choice. (Word of warning: historically, Thrift has not been consistent in their feature support and performance across all the languages, so do some research).

Protocol Buffers vs. Avro, MessagePack

While Thrift and PB differ primarily in their scope, Avro and MessagePack should really be compared in light of the more recent trends: rising popularity of dynamic languages, and JSON over XML. As most every web developers knows, JSON is now ubiquitous, and easy to parse, generate, and read, which explains its popularity. JSON also requires no schema, provides no type checking, and it is a UTF-8 based protocol - in other words, easy to work with, but not very efficient when put on the wire.

MessagePack is effectively JSON, but with efficient binary encoding. Like JSON, there is no type checking or schemas, which depending on your application can be either be a pro or a con. But, if you are already streaming JSON via an API or using it for storage, then MessagePack can be a drop-in replacement.

Avro, on the other hand, is somewhat of a hybrid. In its scope and functionality it is close to PB and Thrift, but it was designed with dynamic languages in mind. Unlike PB and Thrift, the Avro schema is embedded directly in the header of the messages, which eliminates the need for the extra compile stage. Additionally, the schema itself is just a JSON blob - no custom parser required! By enforcing a schema Avro allows us to do data projections (read individual fields out of each record), perform type checking, and enforce the overall message structure.

"The Best" Serialization Format

Reflecting on the use of Protocol Buffers at Google and all of the above competitors it is clear that there is no one definitive, "best" option. Rather, each solution makes perfect sense in the context it was developed and hence the same logic should be applied to your own situation.

If you are looking for a battle-tested, strongly typed serialization format, then Protocol Buffers is a great choice. If you also need a variety of built-in RPC mechanisms, then Thrift is worth investigating. If you are already exchanging or working with JSON, then MessagePack is almost a drop-in optimization. And finally, if you like the strongly typed aspects, but want the flexibility of easy interoperability with dynamic languages, then Avro may be your best bet at this point in time.

HTML5 Visibility API & Page Pre-Rendering

2011-06-25T00:00:00-07:00

Minimizing UI latency is critical for creating a positive user experience - this is true both on the desktop and on the web. A best practice for a "native app" is to decouple the UI and control threads to avoid blocking on any long-running tasks. On the web, things are a lot trickier: our Javascript runtimes are all single-threaded, we can't just spin up an extra thread, and instead have to rely on event-driven programming models. Even worse, unless the invoked computation is local, a roundtrip to the server can easily take hundreds of milliseconds in the network overhead alone.

Not surprisingly, over the course of the past few years we have invented dozens of new Javascript based UI frameworks: all asynchronous, all focused on trying to hide the interaction latency behind a Javascript facade. In theory, all great ideas, but in practice also with their own set of downsides: how about those #!'s, and we have all certainly seen and written a few overly eager setTimeout's which quickly destroy the clients CPU and battery life.

In fact, because the browser VM is a shared resource, we have a classic tragedy of the commons: if every application plays nice, then everyone can have an optimal experience, but the incentives to do so are not clear. Problem is, until very recently we did not even have the tools to address this issue! Page Visibility API is the first HTML5 proposal that is trying to tackle this problem, and browser pre-rendering is also aiming to help us hide some of the network latency in our web applications - let's take a look under the hood.

Browser Pre-fetching vs. Pre-rendering

An average page render requires fetching a dozen resources alongside the actual HTML content. If you dig into your debug console, it is not uncommon to see pages which take on the order of ten seconds to load to completion. Thankfully, the browsers have implemented many tricks to make it seem as if the page is loading much faster - parallel downloads, highly optimized rendering engines, and a never ending battle to speed up Javascript execution. Nonetheless, usually this is still not enough to beat the "native experience".

Well, the server can help us as well: new protocols like SPDY are aiming to reduce the network overhead of fetching multiple resources, and there is even talk of enabling server push of related page assets. Think you can guess what the user may click on next? Firefox 3.5 enabled the pre-fetching API which allows us to hint to the browser what resources it may need to service a subsequent request:

<!--  Specify any & all resources to pre-fetch -->
<link rel="prefetch" href="/images/big.jpg">

<!-- or send an HTTP header -->
Link: </images/big.jpeg>; rel=prefetch

Pre-fetching is a simple optimization, but it requires that we explicitly specify each and every resource - just listing the link of the next HTML page is unlikely to result in a noticeable improvement in the user experience.

This is where the new pre-rendering proposal comes in: instead of specifying a single resource, what if the browser could fetch and render the entire next page, but hide it from you until you click on the link? As of about a month ago, pre-rendering support is in WebKit and Google is already prototyping it with "Instant Pages":

Pre-rendering wins and gotchas

At the moment, the pre-rendering API is limited: only one page can be pre-rendered across the entire VM, and only one page can be put into the pre-render queue per tab. Fetching an entire page taxes both the server and the client, hence you need be sure that you will actually need it. Google's web search team, for example, only enables pre-rendering on search results if they have very high confidence that you may actually click on the result.

Additionally, since we are now pre-rendering the entire page (HTML, CSS, and JS), how does this affect all the interactive content on the page? Knowing nothing about the pre-render step, the requested page can easily pin our CPU, register a pageview and make a request to an ad server for content that the user may never actually see! To solve this problem, WebKit developers have also added the Page Visibility API:

function handleVisibilityChange() {
  if (document.webkitHidden) {
    pausePageJavascript();
  } else {
    startPageJavascript();
  }
}

document.addEventListener("webkitvisibilitychange", handleVisibilityChange, false);

The webkitHidden property tells us the state of the page to solve the original visibility problem, but the webkitvisibilitychange event has another nice side effect: it allows the client to easily detect when a tab is visible, and when it is in the background. Why does this matter? Imagine you have an application which polls the client, or the server, every 50 milliseconds for some updates. With the visibility API, you can gracefully pause or degrade the timer to a much longer poll when the tab is in the background.

Minimizing latency on the web

Both pre-rendering and Page Visibility API's are still in development, but it is great to see more client-side tools to enable web developers to hide the underlying network latency. With these API's, instead of relying on an async Javascript stack, your next multi-step form can be rendered on the server and pre-rendered in a WebKit browser with instant feedback on the client!

Likewise, while browsers like Chrome are already downgrading background tabs in CPU priority, a client-side API to detect foreground tabs is a welcome addition. Let's hope that Firefox, Opera and IE jump on the bandwagon as well!