igvita.com

Innovating with HTTP 2.0 Server Push

2013-06-12T00:00:00-07:00

HTTP 2.0 enables the server to send multiple responses (in parallel) for a single client request - aka, server push. Except, why would we ever want to do such a thing? Well, an average page requires dozens of additional assets, such as JavaScript, CSS, and images, and references to all of these assets are embedded in the very HTML that the server is producing! Hence, instead of waiting for the client to discover references to these resources, what if the server just sent all of them immediately? Server push can eliminate entire roundtrips of unnecessary network latency.

In fact, if you have ever inlined a resource (CSS, JS, or an image), you've been "simulating" server push: an inlined resource is "pushed" as part of the parent document. The only difference is that HTTP 2.0 makes this pattern more efficient and far more powerful!

Hands-on with HTTP 2.0 server push

An inlined resource, by definition, is part of the parent document. Hence, it cannot be cached independently and may need to be duplicated across many different pages - this is inefficient. By contrast, pushed resources are cached individually by the browser and can be reused across many pages. An example is in order:

spdy.createServer(options, function(req, res) {
  // push JavaScript asset (/main.js) to the client
  res.push('/main.js', {'content-type': 'application/javascript'}, function(err, stream) {
    stream.end('alert("hello from push stream!")');
  });

  // write main response body and terminate stream
  res.end('Hello World! <script src="/main.js"></script>');
}).listen(443);

What we have above is a minimal SPDY server implemented with the help of the node-spdy module, which responds to all inbound requests by writing out a "Hello World!" string, followed by script tag. Except, we are also doing something clever: we push the main.js file to the client, the body of which triggers a JavaScript alert.

As a result, by the time the browser discovers the script tag in the HTML response the main.js file is already in cache, and no extra network roundtrips are incurred! HTTP 2.0 server push obsoletes inlining. Best of all, server push is already supported by all the SPDY-capable browsers (Firefox, Opera, and Chrome).

What else can we push?

Replacing inlined resources with server push is the canonical example. However, why stop there, what else could we push? Well, any HTTP response is fair game. Could we push a redirect? Yep, that's a simple task:

spdy.createServer(options, function(req, res) {
  //push JavaScript asset (/newasset.js) to the client
  res.push('/newasset.js', {'content-type': 'application/javascript'}, function(err, stream) {
    stream.end('alert("hello from (redirected) push stream!")');
  });

  // push 301 redirect: /asset.js -> /newasset.js
  res.push('/asset.js', {':status': 301, 'Location': '/newasset.js'}, function(err, stream) {
    stream.end('301 Redirect');
  });

  // write main response body and terminate stream
  res.end('<script src="/asset.js"></script>');
}).listen(443);

Same example as above, except we've changed the asset.js resource to be a 301 redirect to the newasset.js file. The browser caches both the redirect and the asset and executes the script without any extra network roundtrips. Applications? I'll leave it as an exercise for the reader.

Could we take this even further? What about pushing cache invalidations, and revalidations, to the client? We can mark any existing resource in client's cache as stale (i.e. push a stale timestamp), or conversely, update its lifetime by pushing a 304 with a future timestamp. In short, the server could actively manage the client's cache!

If the sever is too aggressive or misbehaved, then the client can limit the number of pushed streams, or cancel individual streams as it desires. Finding the right strategy and balance on both sides will be key to getting the best performance out of server push - not an easy challenge, but one that could yield high returns.

Note: pushing cache revalidations is not supported by current browsers. Should it be?

Client-notifications for server push

HTTP 2.0 server push is not a replacement for technologies such as Server-Sent Events (SSE) or WebSocket. Resources delivered via HTTP 2.0 server push are processed by the browser but do not bubble up to the application code - there is no JavaScript API to get notifications for these events. However, the solution to this dilemma is very simple, because we can combine an SSE channel with server push and get the best of both:

spdy.createServer(options, function(req, res) {
  // set content type for SSE stream
  res.setHeader('Content-Type', 'text/event-stream');

  messageId = 1;
  setInterval(function(){
    // push a simple JSON message into client's cache
    var msg = JSON.stringify({'msg': messageId});
    var resourcePath = '/resource/'+messageId;
    res.push(resourcePath, {}, function(err, stream) { stream.end(msg) });

    // notify client that resource is available in cache
    res.write('data:'+resourcePath+'\n\n');
    messageId+=1;
  }, 2000);
}).listen(443);

Admittedly, a silly example, but one that illustrates the point: the server generates a message on a periodic two second interval, pushes it into the client's cache, and then sends an SSE notification to the client. In turn, the client can subscribe to these events in application code and execute its own logic to process the event:

<script>
  var source = new EventSource('/');

  source.onmessage = function(e) {
    document.body.innerHTML += "SSE notification: " + e.data + '<br />';

    // fetch resource via XHR... from cache!
    var xhr = new XMLHttpRequest();
    xhr.open('GET', e.data);
    xhr.onload = function() {
      document.body.innerHTML += "Message: " + this.response + '<br />';
    };

    xhr.send();
  };
</script>

The long-lived SSE stream, the pushed resources, and all other HTTP 2.0 streams are efficiently multiplexed over a single TCP connection - there is no extra connection overhead or unnecessary roundtrips. By combining SSE and server push we can deliver arbitrary resources (binary or text) to the client, get the benefit of leveraging the native browser cache, and execute the appropriate application logic! Best of all, this works today.

Innovating with server push

Server push opens up an entire world of new optimization opportunities. The examples we have covered above are scratching the surface of what's possible, and there are many additional questions to consider:

Which resources should be pushed and when? Are they in cache already?
Can the server automatically infer which resources should be pushed?
Should advanced applications such as active cache management be supported?
How can we design our applications to get the most out of server push?
...

With HTTP 2.0 the servers have the opportunity to become much, much smarter, both in how they optimize the delivery of individual assets, and even more importantly, the delivery of the entire application. Similarly, the browsers could expose additional API's and capabilities to make this process more efficient. In fact, in the long run, server push may well prove to be the killer feature for HTTP 2.0!

P.S. curious to learn more about HTTP 2.0? Check out the free preview of my O'Reilly book!

Deploying WebP via Accept Content Negotiation

2013-05-01T00:00:00-07:00

Deploying new image formats on the web is a challenge, but not an unsolvable one, and one that would pay high dividends in terms of performance. In fact, there are many ways to tackle the problem: client-side logic, server negotiation, and hybrid strategies. There is a time and place for each one - they are not exclusive.

Having said that, I'm partial to server negotiation as it requires the least amount of work to move the web forward in a significant way: a single flip of a switch by a CDN or a large hosting provider, instantly enabling significant byte savings for all of their users. Computers are great at performing routine negotiation work, humans are not: we get bored, we get lazy, we routinely forget to optimize our images.

WebP deployment checklist

As a hands on example, let's take a look at what we would need to enable Accept negotiation for deploying WebP:

User agents which support WebP should advertise it in the Accept header
Servers and caches should use the Accept header to serve appropriate assets
Origin servers must specify Vary: Accept in generated responses

Most importantly, notice what's missing: there is no mention of having to modify the markup or logic of millions of individual web applications. Having said that, the above steps still require some work, let's dig a bit deeper.

Configuring Accept negotiation in Nginx

The great news is that Chrome Canary is now advertising image/webp in its Accept header for all image requests - with that in place, we can check off the first item on our checklist above. Now, we need to configure our Nginx server to automatically serve the right file. To start, let's assume we have a pre-generated a WebP asset on disk:

location / {
  # check Accept header for webp, check if .webp is on disk
  if ($http_accept ~* "webp") { set $webp T; }
  if (-f $request_filename.webp) { set $webp "${webp}T"; }

  # if WebP is supported, and WebP is on disk, serve it!
  if ($webp = TT) {
    add_header Vary Accept;
    rewrite (.*) $1.webp break;
  }
  # ...
}

First, we check if the Accept header is advertising WebP. Then we check if there is a corresponding file with a .webp extension on disk. If both conditions match, we serve the WebP asset and add Vary: Accept - done.

Now, let's consider the case where Nginx acts as a proxy cache. Here we need to teach Nginx to cache and serve multiple variants of the same image (WebP and non-WebP versions) based on the value of the Accept header:

server {
  location / {
    # lookup inbound requests with exta WebP flag if advertised in Accept
    if ($http_accept ~* "webp") { set $webp T; }
    proxy_cache_key $scheme$proxy_host$request_uri$webp;

    # proxy requests to remote servers
    proxy_pass http://backend;
    proxy_cache my-cache;
  }
}

Not much different from the previous example! Instead of checking the local disk for the asset, we pass the request through to some set of backends, and append the WebP bit to our cache key - that's all there is to it.

Accept fragmentation and Key

Wait, why didn't we just append the $http_accept variable to our proxy cache key above? Unfortunately, the Accept header varies from browser to browser, which means that using the raw header would unnecessarily fragment the cache. Hence, we scan for WebP support in the header and add an additional bit to the cache key.

However, wouldn't it be nice if we had a standard mechanism to define a custom cache key? Well, good news, there is work underway on the new Key response header, which will act as a smart replacement to Vary:

The 'Key' header field for HTTP responses allows an origin server to describe the cache key for a negotiated response: a short algorithm that can be used upon later requests to determine if the same response is reusable.

IE is special, as always!

Unfortunately, Vary support in IE, even in the latest versions, remains very poor: IE does not cache outbound headers, which means it cannot perform the appropriate matching algorithm, and is forced to issue a revalidation request on any asset with Vary. As a result, to avoid the extra requests, we can create a special case for IE:

  if ($http_user_agent ~* "(?i)(MSIE)") {
    # drop Vary for IE users, mark resource as private
    proxy_hide_header Vary;
    add_header Cache-Control private;
  }

With the above configuration in place, IE users won't see Vary, and resource is marked as private: it can still be cached on the client, but it won't be cached by any other intermediate proxy.

The "action plan", continued...

Small steps, but we're heading in the right direction. We've addressed the issue on the client, we know how to configure our servers, now we need to push for better support by proxies and CDN's - ideally, by deploying support for Key! The good news is, this requires work, but it is not rocket science. As a case in point, CDNConnect recently enabled transparent Accept+WebP negotiation for all of their users.

Finally, let's take a look at Facebook's recent experiment with WebP, as summarized by ArsTechnica:

It turns out that Facebook users routinely do things like copy image URLs and save images locally... As a result, users were left with unusable URLs and files; files they couldn't open locally, and URLs that didn't work in Internet Explorer, Firefox, or Safari.

URL sharing is not an issue if Accept negotiation is configured correctly - see instructions above. Next, once a file is saved locally, you need a local WebP codec or viewer. The good news is, Chrome just landed a patch to register itself as a default WebP handler across all platforms - if you have Chrome, you can view local WebP files, and there is a growing list of editors, viewers, and plugins. Also, there are ongoing discussions about enabling the browser to save a "safe copy" of the asset to mitigate the problem entirely. One step at a time, we're getting there!

Update: configuring Apache to serve WebP with Accept negotiation, courtesy of Sergej Müller.

Faster, smaller and more beautiful web with WebP

2013-03-07T00:00:00-08:00

At the risk of sounding repetitive... An average page is now over 1300 kB in size and over 60% of that is in images. Hence, if you have limited time, then inspecting and optimizing your image assets will likely yield the highest rate of return.

Case in point, the new data compression proxy for Chrome applies dozens of different content optimizations, but image optimization almost invariably comes out at the top. End result? On average, data usage is reduced by 50%! The strategy? Simple, transcode all images to WebP!

With that in mind, I had a chance to sit down with Stephen Konig (Product Manager on the WebP team), to chat about the latest news, team progress over last two years, and where WebP is heading. You can scan through the slides, or watch the GDL on YouTube, but below are a few highlights and resources to help you get started.

Focus on adoption in 2013

The primary focus and goal for the WebP team over the course of the past two years has been on developing the necessary features of the format itself - i.e., the engineering part. The good news is, end of 2012 marked an important milestone: support for lossy and lossless compression, alpha channel, animation, metadata, color profiles, and more. With all of these features in place, the focus is shifting towards tooling and driving adoption.

In fact, in the time-honored tradition of dogfooding own products, there is already a large and growing list of Google properties (Gmail, Drive, Picasa, Instant Previews, Play Magazines, Image Search, YouTube, ...) with WebP support. Most recently, Chrome Web Store switched to WebP, saw ~30% byte reduction on average, and is now saving several terabytes of bandwidth per day!

In parallel, there are now over 300,000 sites using the open-source PageSpeed libraries, which enable transparent WebP transcoding on Apache (mod_pagespeed) and Nginx (ngx_pagespeed), and there is a growing list of commercial products (Torbit, EdgeCast) which can do similar optimizations - the bandwidth savings are hard to argue against!

"Time to Glass" vs. Bandwidth

WebP achieves better compression by spending more CPU cycles - that's an inherent tradeoff of any compression algorithm. Today, when compared to JPEG, the encoding speed for WebP is ~10x slower, and decoding is ~1.4x slower when done on the CPU. Is that big deal? The answer depends on your application: if you are generating unique and dynamic images on every request, then the extra CPU overhead is something you'll notice. But if the files are (mostly) static, then the encoding time is a non-issue.

More likely, the concern is not over encoding, but over decoding speeds. Is 1.4x going to hurt your performance? Once again, it depends on your application - as with any performance metric, measure it. The Ebay tech team recently published a great overview of various image optimization techniques:

This test from webpagetest.org compares the page load time of WebP vs. JPEG. The test has one page with 50 images in the WebP format, and another page with the same 50 images in the JPEG format. Because the WebP page had to download fewer bytes (474484 vs. 757228), it completes loading much earlier compared to the JPEG page... If you track your web site's browser usage stats and find that Chrome/Opera users are a sizable chunk, using WebP images will improve the page load time for these users.

Despite the extra decoding time, the visual rendering time is much faster due to fewer bytes shipped. Further, for a significant segment of the population, bytes are (literally) expensive: bandwidth caps are a real constraint for many users, especially on mobile devices and in the developing world.

Deploying WebP on Native and Web platforms

Deploying WebP in a native app, iOS or Android, is actually very straightforward: Android 4.x.x+ has native support for WebP, and there is a backport for earlier versions; on iOS you can use the official libraries provided by the WebP team (tutorial, demo app). Since you control the display logic and the platform, you can safely convert all assets to WebP and save on data transfers both from and to the device - WebP helps for upload cases equally well!

On the web, things get a bit more difficult, but still manageable. Chrome and Opera have native support for WebP, and there is an open discussion with the Firefox team. There are also third party plugins for Safari and Chrome Frame provides support for IE - however, you can't rely on these plugins being present. Instead, for time being you have to fall back to User-Agent detection on the server, or use a JavaScript check - in fact, there is even a JS polyfill! Finally, there is work in progress to fix Accept negotiation to make this entire process much easier.

The most popular technique today and one that is used by PageSpeed and other optimization products, is to rely on server detection: run a User-Agent check, and serve the HTML with WebP image links, or non-WebP links. What are the User-Agent rules? PageSpeed is open-source, so the answer is right in the code. Further, to simplify this process, I've translated the rules into sample configuration files for Varnish and Nginx:

webp-detect.git - User-Agent detection rules for WebP

With above detection in place, Varnish and Nginx will report an extra WebP: lossy, lossless header to the app server, allowing your application to generate customized HTML based on available WebP support. In turn, the static image assets can be safely cached either on the local file system or on any CDN service. The only other caveat is that the HTML must be marked with Cache-Control: private to ensure that an intermediate cache does not accidentally serve the wrong file to the wrong browser.

For more details on WebP, scan through the slides, and check out the GDL on YouTube for additional context.

High Performance Networking in Google Chrome

2013-01-31T00:00:00-08:00

The following is a draft chapter for the upcoming "The Performance of Open Source Applications" (POSA), a sequel to The Architecture of Open Source Applications. POSA is a collection of essays about performance optimization, designing for performance, managing performance as part of a development process, and more. The book will be published in Spring 2013 under a Creative Commons license with royalties going to Amnesty International.

History and guiding principles of Google Chrome
The many facets of performance
What is a modern web application?
The life of a resource request on the wire
What is "fast enough"?
Chrome's network stack from 10,000 feet
Lifetime of your browser session...
Chrome gets faster as you use it

History and guiding principles of Google Chrome # ↑

Google Chrome was first released in the second half of 2008, as a beta version for the Windows platform. The Google-authored code powering Chrome was also made available under a permissive BSD license - aka, the Chromium project. To many observers, this turn of events came as a surprise: the return of the browser wars? Could Google really do much better?

"It was so good that it essentially forced me to change my mind..." - Eric Schmidt, on his initial reluctance to the idea of developing Google Chrome.

Turns out, they could. Today Chrome is one of the most widely used browsers on the web (35%+ of the market share according to StatCounter) and is now available on Windows, Linux, OS X, Chrome OS, as well as Android and iOS platforms. Clearly, the features and the functionality resonated with the users, and many innovations of Chrome have also found their way into other popular browsers.

The original 38-page comic book explanation of the ideas and innovations of Google Chrome offers a great overview of the thinking and design process behind the popular browser. However, this was only the beginning. The core principles that motivated the original development of the browser continue to be the guiding principles for ongoing improvements in Chrome:

Speed: the objective is to make the fastest browser
Security: provide the most secure environment to the user
Stability: provide a resilient and stable web application platform
Simplicity: sophisticated technology, wrapped in a simple user experience

As the team observed, many of the sites we use today aren't just web pages, they are applications. In turn, the ever more ambitious applications require speed, security, and stability. Each of these deserves its own dedicated chapter, but since our subject is performance, our focus will be primarily on speed.

A modern browser is a platform, just like your operating system, and Google Chrome is designed as such. Prior to Google Chrome, all major browsers were built as a monolithic, single process applications. All open pages shared the same address space and contended for the same resources. A bug in any page, or the browser, ran the risk of compromising the entire experience.

By contrast, Chrome works on a multi-process model, which provides process and memory isolation, and a tight security sandbox for each tab. In an increasingly multi-core world, the ability to isolate the processes as well as shield each open tab from other misbehaving pages has by itself proven to give Chrome a significant performance edge over the competition. In fact, it is important to note that most other browsers have followed suit, or are in the process of migrating to similar architecture.

With an allocated process in place, the execution of a web program primarily involves three tasks: fetching resources, page layout and rendering, and JavaScript execution. The rendering and script steps follow a single-threaded, interleaved model of execution - it is not possible to perform concurrent modifications of the resulting Document Object Model (DOM). This is in part due to the fact that JavaScript itself is a single threaded language. Hence, optimizing how the rendering and script execution runtimes work together is of critical importance, both to the web developers building the applications as well as the developers working on the browser.

For rendering, Chrome uses WebKit, which is a fast, open-source, and standards compliant layout engine. For JavaScript, Chrome ships with its own, heavily optimized "V8" JavaScript runtime, which was also released as a standalone open-source project and has found its way into many other popular projects - e.g., runtime for node.js. However, optimizing V8 JavaScript execution, or the WebKit parsing and rendering pipelines won't do much good if the browser is blocked on the network, waiting for the resources to arrive!

The ability of the browser to optimize the order, priority, and latency of each network resource is one of the most critical contributors to the overall user experience. You may not be aware of it, but Chrome's network stack is, quite literally, getting smarter every day, trying to hide or decrease the latency cost of each resource: it learns likely DNS lookups, it remembers the topology of the web, it preconnects to likely destination targets, and more. From the outside, it presents itself as a simple resource fetching mechanism, but from the inside it is an elaborate and a fascinating case study for how to optimize web performance and deliver the best experience to the user.

Let's dive in...

What is a modern web application? # ↑

Before we get to the tactical details of how to optimize our interaction with the network, it helps to understand the trends and the landscape of the problem we are up against. In other words, what does a modern web page, or application look like?

The HTTP Archive project tracks how the web is built, and it can help us answer this question. Instead of crawling the web for the content, it periodically crawls the most popular sites to record and aggregate analytics on the number of used resources, content types, headers, and other metadata for each individual destination. The stats, as of January 2013, may surprise you. An average page, amongst the top 300,000 destinations on the web is:

1280 KB in size
composed of 88 resources
connects to 15+ distinct hosts

Let that sink in. Over 1MB in size on average, composed of 88 resources such as images, JavaScript, and CSS, and delivered from 15 different own and third-party hosts! Further, each of these numbers have been steadily increasing over the past few years, and there are no signs of stopping. We are increasingly building larger and more ambitious web applications.

Applying basic math to the HTTP Archive numbers reveals that an average resource is about 12KB in size (1045 KB / 84 resources), which means that most network transfers in the browser are short and bursty. This presents its own set of complications because the underlying transport (TCP) is optimized for large, streaming downloads. Let's peel back the onion and inspect one of these network requests...

The life of a resource request on the wire # ↑

The W3C Navigation Timing specification provides a browser API and visibility into the timing and performance data behind the life of every request in the browser. Let's inspect the components, as each is a critical piece of delivering the optimal user experience:

Given the URL of a resource on the web, the browser starts by checking its local and application caches. If you have previously fetched the resource and the appropriate cache headers were provided (Expires, Cache-Control, etc.), then it is possible that we may be allowed to use the local copy to fulfill the request - the fastest request is a request not made. Alternatively, if we have to revalidate the resource, if it expired, or if we simply haven't seen it before, then a costly network request must be dispatched.

Given a hostname and resource path, Chrome first checks for existing open connections it is allowed to reuse - sockets are pooled by {scheme, host, port}. Alternatively, if you have configured a proxy, or specified a proxy auto-config (PAC) script, then Chrome checks for connections through the appropriate proxy. PAC scripts allow for different proxies based on URL, or other specified rules, each of which can have its own socket pool. Finally, if neither of the above conditions is matched, then the request must begin by resolving the hostname to its IP address - aka, a DNS lookup.

If we are lucky, the hostname may already be cached in which case the response is usually just one quick system call away. If not, then a DNS query must be dispatched before any other work can happen. The time taken to do the DNS lookup will vary based on your internet provider, the popularity of the site and the likelihood of the hostname to be in intermediate caches, as well as the response time of the authoritative servers for that domain. In other words, there are a lot of variables at play, but it's not unusual for a DNS lookup to take up to several hundred milliseconds - ouch.

With the resolved IP address in hand, Chrome can now open a new TCP connection to the destination, which means that we must perform the "three-way handshake": SYN > SYN-ACK > ACK. This exchange adds a full roundtrip of latency delay to each and every new TCP connection - no shortcuts. Depending on the distance between the client and the server, as well as the chosen routing path, this can yield from tens to hundreds, or even thousands, of milliseconds of delay. All of this work and latency is before even a single byte of application data has hit the wire!

Once the TCP handshake is complete, and if we're connecting to a secure destination (HTTPS), then the SSL handshake must take place. This can add up to two additional roundtrips of latency delay between client and server. If the SSL session is cached, then we can "escape" with just one additional roundtrip.

Finally, Chrome is able to dispatch the HTTP request (requestStart in the Nav Timing figure above). Once received, the server can process the request and then stream the response data back to the client. This incurs a minimum of a network roundtrip, plus the processing time on the server. Following that, we're done. Well, that is unless the actual response is an HTTP redirect! In which case, we may have to repeat the entire cycle once over. Have a few gratuitous redirects on your pages? You may want to revisit that decision!

Have you been counting all the delays? To illustrate the problem, let's assume the worst case scenario for a typical broadband connection: local cache miss, followed by a relatively fast DNS lookup (50 ms), TCP handshake, SSL negotiation, and a relatively fast (100 ms) server response time, with a round-trip time of 80 ms (an average round-trip across continental USA):

50ms for DNS
80ms for TCP handshake (one RTT)
160ms for SSL handshake (two RTT's)
40ms for request to server
100ms for server processing
40ms for response from the server

That's 470 milliseconds for a single request, which translates to over 80% of network latency overhead as compared to the actual server processing time to fulfill the request - we have some work to do here! In fact, even 470 milliseconds may be an optimistic estimate:

If the server response does not fit into the initial TCP congestion window (4-15 KB), then one or more additional roundtrips of latency is introduced
SSL delays could get even worse if we need to fetch a missing certificate or perform an online certificate status check (OCSP), both of which will require an entirely new TCP connection, which can add hundreds and even thousands of milliseconds of additional latency

What is "fast enough"? # ↑

The network overhead of DNS, handshakes, and the roundtrip times is what dominates the total time in our earlier case - the server response time accounts for only 20% of the total latency! But, in the grand scheme of things, do these delays even matter? If you are reading this, then you probably already know the answer: yes, very much so.

Past user experience research paints a consistent picture in what we, as users, expect in terms of responsiveness of any applications, both offline and online:

Delay	User Reaction
0 - 100ms	Instant
100 - 300ms	Small perceptible delay
300 - 1000ms	Machine is working
1s+	Mental context switch
10s+	I'll come back later...

The table above also explains the unofficial rule of thumb in the web performance community: render your pages, or at the very least, provide visual feedback in under 250 ms to keep the user engaged. This is not speed simply for speed's sake. Studies at Google, Amazon, Microsoft, as well as thousands of other sites show that additional latency has a direct impact on the bottom line of your site: faster sites yield more pageviews, higher engagement from the users, and see higher conversion rates.

So, there you have it, our optimal latency budget is 250 ms, and yet as we saw in the example above, the combination of a DNS lookup, the TCP and SSL handshakes, and propagation times for the request add up to 370 ms. We're 50% over budget, and we still haven't factored in the server processing time!

To most users and even web-developers, the DNS, TCP, and SSL delays are entirely transparent and are negotiated at network layers to which few of us descend or think about. And yet, each of these steps is critical to the overall user experience, since each extra network request can add tens or hundreds of milliseconds of latency. This is the reason why Chrome's network stack is much, much more than a simple socket handler.

Now that we've identified the problem, let's dive into the implementation details...

Chrome's network stack from 10,000 feet # ↑

Multi-process architecture # ↑

Chrome's multi-process architecture carries important implications for how each network request is handled within the browser. Under the hood, Chrome actually supports four different execution models that determine how the process allocation is performed.

By default, desktop Chrome browsers use the process-per-site model, that isolates different sites from each other, but groups all instances of the same site into the same process. However, to keep things simple, let's assume one of the simplest cases: one distinct process for each open tab. From the network performance perspective, the differences here are not substantial, but the process-per-tab model is much easier to understand.

The architecture dedicates one render process to each tab, which itself contains an instance of the WebKit open-source layout engine for interpreting and layout out the HTML (aka, "HTML Renderer" in the diagram), an instance of the V8 JavaScript engine, and the glue code to bridge these and a few other components. If you are curious, the Chromium wiki contains a great introduction to the plumbing.

Each of these "render" processes is executed within a sandboxed environment that has limited access to the user's computer - including the network. To gain access to these resources, each render process communicates with the browser (kernel) process, which is able to impose security and access policies on each renderer.

Inter-process communication (IPC) and Multi-process resource loading # ↑

All communication between the renderer and the kernel process in Chrome is done via IPC. On Linux and OSX, a socketpair() is used, which provides a named pipe transport for asynchronous communication. Each message from the renderer is serialized and passed to a dedicated I/O thread, which dispatches it to the kernel process. On the receiving end, the kernel process provides a filter interface, which allows Chrome to intercept resource IPC requests (see ResourceMessageFilter) which should be handled by the network stack.

One of the advantages of this architecture is that all resource requests are handled entirely on the I/O threads and neither any UI generated activity, or network events interfere with each other. The resource filter runs in the I/O thread in the browser process, intercepts the resource request messages, and forwards them to a ResourceDispatcherHost singleton in the browser process.

The singleton interface allows the browser to control each renderer's access to the network, but it also enables efficient, and consistent resource sharing:

Socket pool and connection limits: the browser is able to enforce limits on the number of open sockets per profile (256), proxy (32), and {scheme, host, port} (6) groups. Note that this allows up to six HTTP and six HTTPS connections to the same {host, port}!
Socket reuse: persistent TCP connections are retained in the socket pool for some time after servicing the request to enable connection reuse, which avoids the extra DNS, TCP, and SSL (if required) setup overhead imposed on each new connection.
Socket late-binding: requests are associated with an underlying TCP connection only once the socket is ready to dispatch the application request, allowing better request prioritization (e.g., arrival of a higher priority request while the socket was connecting), better throughput (e.g., re-use of a "warm" TCP connection in cases where an existing socket becomes available while a new connection is being opened), as well as a general-purpose mechanism for TCP pre-connect, and a number of other optimizations.
Consistent session state: authentication, cookies, and cached data is shared between all render processes.
Global resource and network optimizations: the browser is able to make decisions across all render processes and outstanding requests. For example, giving network priority to the requests initiated by the foreground tab.
Predictive optimizations: by observing all network traffic, Chrome is able to build and refine predictive models to improve performance.
... and the list goes on.

As far as the render process is concerned, it is simply sending a resource request message over IPC, which is tagged with a unique request ID to the browser process, and the browser kernel process handles the rest.

Cross-platform resource fetching # ↑

One of the chief concerns in the implementation of Chrome's network stack is portability across many different platforms: Linux, Windows, OS X, Chrome OS, Android, and iOS. To address this challenge, the network stack is implemented as a mostly single-threaded (there are separate cache and proxy threads), cross-platform library, which allows Chrome to reuse the same infrastructure and provide the same performance optimizations, as well as a greater opportunity for optimization across all platforms.

All of the network code is, of course, open source and can be found in the "src/net" subdirectory. We won't examine each component in detail, but the layout of the code itself tells you a lot about its capabilities and structure. A few examples:

net/android	Bindings to the Android runtime
net/base	Common net utilities, such as host resolution, cookies, network change detection, and SSL certificate management
net/cookies	Implementation of storage, management, and retrieval of HTTP cookies
net/disk_cache	Disk and memory cache implementation for web resources
net/dns	Implementation of an asynchronous DNS resolver
net/http	HTTP protocol implementation
net/proxy	Proxy (SOCKS and HTTP) configuration, resolution, script fetching, ...
net/socket	Cross-platform implementations of TCP sockets, SSL streams, and socket pools
net/spdy	SPDY protocol implementation
net/url_request	URLRequest, URLRequestContext, and URLRequestJob implementations
net/websockets	WebSockets protocol implementation

Each of the above makes for a great read for the curious - the code is well documented, and you'll find plenty of unit tests for every component.

Architecture and performance on mobile platforms # ↑

Mobile browser usage is growing at an exponential rate and even by modest projections, it will eclipse desktop browsing in the not so distant future. Needless to say, delivering an optimized mobile experience has been a top priority for the Chrome team. In early 2012, Chrome for Android was announced, and a few months later, Chrome for iOS followed.

The first thing to note about the mobile version of Chrome, is that it's not simply a direct adaptation of the desktop browser - that would not deliver the best user experience. By its very nature, the mobile environment is both much more resource constrained, and has many fundamentally different operating parameters:

Desktop users navigate with the mouse, may have overlapping windows, have a large screen, are mostly not power constrained, usually have a much more stable network connection, and have access to much larger pools of storage and memory.
Mobile users use touch and gesture navigation, have a much smaller screen, are battery and power constrained, are often on metered connections, and have limited local storage and memory.

Further, there is no such thing as a "typical mobile device". Instead there is a wide range of devices with varying hardware capabilities, and to deliver the best performance, Chrome must to adapt to the operating constraints of each and every device. Thankfully, the various execution models allow Chrome to do exactly that!

On Android devices, Chrome leverages the same multi-process architecture as the desktop version - there is a browser process, and one or more renderer processes. The one difference is that due to memory constraints of the mobile device, Chrome may not be able to run a dedicated renderer for each open tab. Instead, Chrome determines the optimal number of renderer processes based on available memory, and other constraints of the device, and shares the renderer process between the multiple tabs.

In cases where only minimal resources are available, or if Chrome is unable to run multiple processes, it can also switch to use a single-process, multi-threaded processing model. In fact, on iOS devices, due to sandboxing restrictions of the underlying platform, it does exactly that - it runs a single, but multi-threaded process.

What about network performance? First off, Chrome uses the same network stack on Android and iOS, as it does on all other versions. This enables all of the same network optimizations across all platforms, which gives Chrome a significant performance advantage. However, what is different, and is often adjusted based on the capabilities of the device and the network in use, are variables such as priority of speculative optimization techniques, socket timeouts and management logic, cache sizes, and more.

For example, to preserve battery, mobile Chrome may opt-in to use lazy closing of idle sockets - sockets are closed only when opening new ones to minimize radio use. Similarly, since prerendering (which we will discuss below), may require significant network and processing resources, it is often only enabled when the user is on Wi-Fi.

Optimizing the mobile browsing experience is one of the highest priority items for the Chrome development team, and we can expect to see a lot of new improvements in the months and years to come. In fact, it is a topic that deserves its own separate chapter - perhaps in the next installment of the POSA series!

Speculative optimization with Chrome's Predictor # ↑

Chrome gets faster as you use it. This feat is accomplished with the help of a singleton Predictor object, which is instantiated within the browser kernel process, and whose sole responsibility is to observe network patterns and to learn and anticipate likely user actions in the future. A few example signals processed by the Predictor include:

Users hovering their mouse over a link is a good indicator of a likely, upcoming navigation event, which Chrome can help accelerate by dispatching a speculative DNS lookup of the target hostname, as well as potentially starting the TCP handshake. By the time the user clicks, which takes ~200 ms on average, there is a good chance that we have already completed the DNS and TCP steps, allowing us to eliminate hundreds of milliseconds of extra latency for the navigation event.
Typing in the Omnibox (URL) bar triggers high-likelihood suggestions, which may similarly kick off a DNS lookup, TCP pre-connect, and can even pre-render the page in a hidden tab!
Each one of us has a list of favorite sites that we visit every day. Chrome can learn the subresources on these sites and speculatively pre-resolve and perhaps even pre-fetch them to accelerate the browsing experience. And the list goes on...

Chrome learns the topology of the web, as well as your own browsing patterns, as you use it. If it does the job well, it can eliminate hundreds of milliseconds of latency from each navigation and get the user closer to the holy grail of the "instant page load". To achieve this goal, Chrome leverages four core optimization techniques:

DNS pre-resolve	resolve hostnames ahead of time, to avoid DNS latency
TCP pre-connect	connect to destination server ahead of time, to avoid TCP handshake latency
Resource prefetching	fetch critical resources on the page ahead of time, to accelerate rendering of the page
Page prerendering	fetch the entire page with all of its resources ahead of time, to enable instant navigation when triggered by the user

Each decision to invoke one or several of these techniques is optimized against a large number of constraints. After all, each is a speculative optimization, which means that if done poorly, it might trigger unnecessary work and network traffic, or even worse, have a negative effect on the loading time for an actual navigation triggered by the user.

How does Chrome address this problem? The predictor consumes as many signals as it can, which include user generated actions, historical browsing data, as well as signals from the renderer and the network stack itself.

Not unlike the ResourceDispatcherHost, which is responsible for coordinating all of the network activity within Chrome, the Predictor object creates a number of filters on user and network generated activity within Chrome:

IPC channel filter to monitor for signals from the render processes
ConnectInterceptor object is added to each request, such that it can observe the traffic patterns and record success metrics for each request

As a hands on example, the render process can trigger a message to the browser process with any of the following hints, which are conveniently defined in ResolutionMotivation (url_info.h):

enum ResolutionMotivation {
  MOUSE_OVER_MOTIVATED,     // Mouse-over initiated by the user.
  OMNIBOX_MOTIVATED,        // Omni-box suggested resolving this.
  STARTUP_LIST_MOTIVATED,   // This resource is on the top 10 startup list.
  EARLY_LOAD_MOTIVATED,     // In some cases we use the prefetcher to warm up
                            // the connection in advance of issuing the real
                            // request.

  // The following involve predictive prefetching, triggered by a navigation.
  // The referring_url_ is also set when these are used.
  STATIC_REFERAL_MOTIVATED,  // External database suggested this resolution.
  LEARNED_REFERAL_MOTIVATED, // Prior navigation taught us this resolution.
  SELF_REFERAL_MOTIVATED,    // Guess about need for a second connection.

  // <snip> ...
};

Given such a signal, the goal of the predictor is to evaluate the likelihood of its success, and then to trigger the activity if resources are available. Every hint may have a likelihood of success, a priority, and an expiration timestamp, the combination of which can be used to maintain an internal priority queue of speculative optimizations. Finally, for every dispatched request from within this queue, the predictor is also able to track its success rate, which allows it to further optimize its future decisions.

Chrome network architecture in a nutshell # ↑

Chrome uses a multi-process architecture, which isolates render processes from the browser process
Chrome maintains a single instance of the resource dispatcher, which is shared across all render processes, and runs within the browser kernel process
The network stack is a cross-platform, (mostly) single-threaded library
The network stack uses non-blocking operations to manage all network operations
Shared network stack allows efficient resource prioritization, reuse, and provides the browser with ability to perform global optimization across all running processes
Each render process communicates with the resource dispatcher via IPC
Resource dispatcher intercepts resource requests via a custom IPC filter
Predictor intercepts resources request and response traffic to learn and optimize future network requests
Predictor may speculatively schedule DNS, TCP, and even resource requests based on learned traffic patterns, saving hundreds of milliseconds when the navigation is triggered by the user

Lifetime of your browser session... # ↑

With the 10,000 foot architecture view of the Chrome network stack in mind, let's now take a closer look at the kinds of user-facing optimizations enabled within the browser. Specifically, let's imagine we have just created a new Chrome profile and are ready to start our day.

Optimizing the cold-boot experience # ↑

The first time you load your browser, it of course knows little about your favorite sites or navigation patterns. But, as it turns out, many of us follow the same routine after a cold-boot of the browser, where we may navigate to our email inbox, favorite news site, a social site, an internal portal, and so on. The specific sites will, of course, vary, but the similarity of all these sessions allows the Chrome predictor to accelerate your cold-boot experience!

Chrome remembers the top ten likely hostnames accessed by the user following the browser start - note that this is not the top ten global destinations, but specifically the destinations following a fresh browser start. As the browser loads, Chrome can trigger a DNS pre-fetch for the likely destinations! If you are curious, you can inspect your own startup hostname list by opening a new tab and navigating to chrome://dns. At the top of the page, you will find the list of the top ten likely startup candidates for your profile.

Above screenshot is an example from my own Chrome profile. How do I usually begin my browsing? Frequently by navigating to Google Docs if I'm working on an article such at this one. Not surprisingly, we see a lot of Google hostnames in the list!

Optimizing interactions with the Omnibox # ↑

One of the innovations of Chrome was the introduction of the Omnibox, which unlike its predecessors handles much more than just destination URLs. Besides remembering the URLs of pages that the user visited in the past, it also offers full text search over your history (tip: instead of the URL, try typing the name of the page you've recently visited), as well as a tight integration with the search engine of your choice.

As the user types, the Omnibox automatically proposes an action, which is either a URL based on your navigation history, or a search query. Under the hood, each proposed action is scored with respect to the query, as well as its past performance. In fact, Chrome allows us to inspect this data by visiting chrome://predictors.

Chrome maintains a history of the user entered prefixes, the actions it has proposed, as well as the hit rate for each one. For my own profile, you can see that whenever I enter "g" in the Omnibox, there is a 76% chance that I'm heading to Gmail. Once I add an "m" (for "gm"), then the confidence rises to 99.8% - in fact, out of the 412 recorded visits, I didn't end up going to Gmail, after entering "gm" only once!

But, you're thinking, what does this have to do with the network stack? Well, the yellow and green colors for the likely candidates are also important signals for the ResourceDispatcher! If we have a likely candidate (yellow), Chrome may trigger a DNS pre-fetch for the target host. If we have a high confidence candidate (green), then Chrome may also trigger a TCP pre-connect once the hostname has been resolved. And finally, if both complete while the user is still deliberating, then Chrome may even pre-render the entire page in a hidden tab.

Alternatively, if there is no good match for the entered prefix based on past navigation history, then Chrome may issue a DNS pre-fetch and TCP pre-connect to your search provider, in anticipation of a likely search request.

An average user takes hundreds of milliseconds to fill in their query and to evaluate the proposed autocomplete suggestions. In the background, Chrome is able to pre-fetch, pre-connect, and in certain cases even pre-render the page, such that by the time the user is ready to hit the "enter" key, much of the network latency has already been eliminated!

Optimizing cache performance # ↑

The best, and the fastest request, is a request not made. Whenever we talk about performance, we would be amiss if we didn't talk about the cache -- you are providing Expires, ETag, Last-Modified, and Cache-Control response headers for all the resources on your pages, right? If not, stop, go fix it, we'll wait.

Chrome has two different implementations of the internal cache: one backed by local disk, and second which stores everything in memory. The in-memory implementation is used for the Incognito browsing mode and is wiped clean whenever you close the window. Both implement the same internal interface (disk_cache::Backend, and disk_cache::Entry), which greatly simplifies the architecture - and if you are so inclined, allows you to easily experiment with your own, experimental cache implementations.

Internally, the disk cache implements its own set of data structures, all of which are stored within a single cache folder for your profile. Inside this folder, there are index files, which are memmapped when the browser starts, and data files which store the actual data, alongside the HTTP headers and other bookkeeping information. As an interesting sidenote, resources up to 16KB in size are stored in shared data block-files, and larger files get their own dedicated files on disk. Finally, for eviction, the disk cache maintains an LRU which uses ranking metrics such as frequency of access and age of resource into account.

If you are ever curious about the state of the Chrome cache, open a new tab and navigate to chrome://net-internals/#httpCache. Alternatively, if you want to see the actual HTTP metadata and the cached response, you can also visit chrome://cache, which will enumerate all of the resources currently available in the cache. From that page, search for a resource you're looking for and click on the URL to see the exact, cached headers and response bytes.

Optimizing DNS with prefetching # ↑

We have already mentioned DNS pre-resolution on several occasions, so before we dive into the implementation, let's review the cases in which it may be triggered, and why:

The WebKit document parser, which runs in the render process, may provide a list of hostnames for all the links on the current page, which Chrome may, or may not choose to pre-resolve ahead of time.
Renderer process may trigger a mouse hover or "button down" event as an early signal of user's intent to perform a navigation.
The Omnibox may trigger a resolve request based on a high likelihood suggestion.
Chrome predictor may request hostname resolution based on past navigation and resource request data - more on this below.
The owner of the page may explicitly indicate to Chrome which hostnames it should pre-resolve.

In all cases, DNS pre-resolution is treated as a hint. Chrome does not guarantee that the pre-resolution will occur, rather it uses each signal in combination with its own predictor to assess the hint and decide on a course of action. In the "worst case", if we weren't able to pre-resolve the hostname in time, the user would have to wait for an explicit DNS resolution, followed by TCP connection time, and finally the actual resource fetch. However, when this occurs, the predictor can take note and adjust its future decisions accordingly - it gets faster, and smarter, as you use it.

One of the optimizations we have not covered previously is the ability of Chrome to learn the topology of each site and then use this information to accelerate future visits. Specifically, recall that an average page consists of 88 resources, which are delivered from 30+ distinct hosts. Well, each time you perform a navigation, Chrome may record the hostnames for the popular resources on the page, and during a future visit, it may choose to trigger a DNS pre-resolve and even a TCP pre-connect for some, or all of them!

To inspect the subresource hostnames stored by Chrome, navigate to chrome://dns and search for any popular destination hostname for your profile. In the example above, you can see the six subresource hostnames that Chrome remembered for Google+, as well as stats for the number of cases when a DNS pre-resolution was triggered, or a TCP pre-connect was performed, as well as an expected number of requests that will be served by each. This internal accounting is what enables the Chrome predictor to perform its optimizations.

In addition to all of the internal signals, the owner of the site is also able to embed additional markup on their pages to request the browser to pre-resolve a hostname:

  <link rel="dns-prefetch" href="//host_name_to_prefetch.com">

Why not simply rely on the automated machinery in the browser? In some cases, you may want to pre-resolve a hostname which is not mentioned anywhere on the page. The canonical example is, of course, redirects: a link may point to a host, like an analytics tracking service, which then redirects the user to the actual destination. By itself, Chrome cannot infer this pattern, but you can help it by providing a manual hint and get the browser to resolve the hostname of the actual destination ahead of time.

So, how is this all implemented under the hood? The answer to this question, just like all other optimizations we have covered, depends on the version of Chrome, since the team is always experimenting with new and better ways to improve performance. However, broadly speaking, the DNS infrastructure within Chrome has two major implementations: historically, Chrome has relied on the platform-independent getaddrinfo() system call, and delegated the actual responsibility for the lookups to the operating system, however this approach is in the process of being replaced with Chrome's own implementation of an asynchronous DNS resolver.

The original implementation, which relied on the operating system, has its benefits: less and simpler code, and the ability to leverage the operating system's DNS cache. However, getaddrinfo() is also a blocking system call, which meant that Chrome had to create and maintain a dedicated worker thread-pool to allow it to perform multiple lookups in parallel. This unjoined pool was capped at six worker threads, which is an empirical number based on lowest common denominator of hardware - turns out, higher numbers of parallel requests can overload some users' routers!

For pre-resolution with the worker-pool, Chrome simply dispatches the getaddrinfo() call, which blocks the worker thread until the response is ready, at which point it just discards the returned result and begins processing the next prefetch request. Discards it? The result is cached by the OS DNS cache, which returns an immediate response to future, actual getaddrinfo() lookups. Simple, effective, works well enough in practice.

Well, effective, but not good enough! The getaddrinfo() call hides a lot of useful information from Chrome, such as the time-to-live (TTL) timestamps for each record, as well as the state of the DNS cache itself. To improve performance, Chrome team decided to implement their own, cross-platform, asynchronous DNS resolver.

By moving DNS resolution into Chrome the new async resolver enables a number of new optimizations:

better control of retransmission timers, and ability to execute multiple queries in parallel
visibility into record TTLs, which allows Chrome to refresh popular records ahead of time
better behavior for dual stack implementations (IPv4 and IPv6)
failovers to different servers, based on RTT or other signals

All of the above, and more, are ideas for continuous experimentation and improvement within Chrome. Which brings us to the obvious question: how do we know and measure the impact of these ideas? Simple, Chrome tracks detailed network performance stats and histograms for each individual profile. To inspect the collected DNS metrics, open a new tab, and head to chrome://histograms/DNS.

The above histogram shows the distribution of latencies for DNS prefetch requests: roughly 50% (rightmost column) of the prefetch queries were finished within 20ms (leftmost column). Note that this is data based on a recent browsing session (9869 samples), and is private to the user. If the user has opted in to report their usage stats in Chrome, then the summary of this data is anonymized and periodically beaconed back to the engineering team, which is then able to see the impact of their experiments and adjust accordingly. Rinse, lather, repeat.

Optimizing TCP connection management with pre-connect # ↑

We have pre-resolved the hostname and we have a high likelihood navigation event that's about to happen, as estimated by the Omnibox, or the Chrome predictor. Why not go one step further, and also speculatively pre-connect to the destination host and complete the TCP handshake before the user dispatches the request? By doing so, we can eliminate another full round-trip of latency delay, which can easily save hundreds of milliseconds for the user. Well, that's exactly what TCP-preconnect is and how it works!

To see the hosts for which a TCP preconnect has been triggered, open a new tab and visit chrome://dns.

First, Chrome checks its socket pools to see if there is an available socket for the hostname, which it may be able to reuse - keep-alive sockets are kept in the pool for some period of time, to avoid the TCP handshake and slow-start penalties. If no socket is available, then it can initiate the TCP handshake, and place it in the pool. Then, when the user initiates the navigation, the HTTP request can be dispatched immediately.

Curious to see the state of all the open sockets in Chrome? Simple, head to: chrome://net-internals#sockets

Note that you can also drill into each socket and inspect the timeline: connect and proxy times, arrival times for each packet, and more. Last but not least, you can also export this data for further analysis or a bug report. Having good instrumentation is key to any performance optimization, and chrome://net-internals is the nexus of all things networking in Chrome - if you haven't explored it yet, you should!

Optimizing resource loading with prefetch hints # ↑

Sometimes, the author of a page is able to provide additional navigation, or page context, based on the structure or the layout of their site, and help the browser optimize the experience for the user. Chrome supports two such hints, which can be embedded in the markup of the page:

  <link rel="subresource" href="/javascript/myapp.js">
  <link rel="prefetch"    href="/images/big.jpeg">

Subresource and prefetch look very similar, but have very different semantics. When a link resource specifies its relationship as "prefetch", it is an indication to the browser that this resource might be needed in a future navigation. In other words, effectively it is a cross-page hint. By contrast, when a resource specifies the relationship as a "subresource", it is an early indication to the browser that the resource will be used on a current page, and that it may want to dispatch the request before it encounters it later in the document.

As you would expect, the different semantics of the hints lead to very different behavior by the resource loader. Resources marked with prefetch are considered low priority and might be downloaded by the browser only once the current page has finished loading. Whereas subresource resources are fetched with high priority as soon as they are encountered and will compete with the rest of the resources on the current page.

Both hints, when used well and in the right context, can help significantly with optimizing the user experience on your site. Finally, it is also important to note that prefetch is part of the HTML5 spec, and as of today supported by Firefox and Chrome, whereas subresource is currently only available in Chrome.

Optimizing resource loading with browser prefreshing # ↑

Unfortunately, not all site owners are able or willing to provide the browser with subresource hints in their markup. Further, even if they do, we must wait for the HTML document to arrive from the server before we are able to parse the hints and begin fetching the necessary subresources - depending on the server response time, as well as the latency between the client and the server, this could take hundreds and even thousands of milliseconds.

However, as we saw earlier, Chrome is already learning the hostnames of the popular resources to perform DNS pre-fetching. So, why couldn't it do the same, but go one step further and perform the DNS lookup, use TCP preconnect, and then also speculatively prefetch the resource? Well, that's exactly what "prefreshing" could do:

User initiates a request to a target URL
Chrome queries its Predictor for learned subresources associated with target URL and initiates the sequence of DNS prefetch, TCP preconnect, and resource prefreshing
If the learned subresource is in the cache, then its loaded from disk and into memory
If the learned subresource is missing, or has expired, then a network request is made

Resource prefreshing is a great example of the workflow of every experimental optimization in Chrome - in theory, it should enable better performance, but there are many tradeoffs as well. There is only way to reliably determine if it will make the cut and make it into Chrome: implement it and run it as A/B experiment in some of the pre-release channels with real users, on real networks, with real browsing patterns.

As of early 2013, the Chrome team is in the early stages of discussing the implementation. If it makes the cut based on gathered results, we may see prefreshing in Chrome sometime later in the year. The process of improving Chrome network performance never stops, the team is always experimenting with new approaches, ideas, and techniques.

Optimizing navigation with prerendering # ↑

Each and every optimization we have covered up to now helps reduce the latency between the user's direct request for a navigation and the resulting page rendering in their tab. However, what would it take to have a truly instant experience? Based on the UX data we saw earlier, this interaction would have to happen in less than 100 milliseconds, which doesn't leave much room for network latency at all. What could we do to deliver a rendered page in sub 100 milliseconds?

Of course, you already know the answer, since this is a common pattern employed by many users: if you open multiple tabs then switching between tabs is instant and is definitely much faster than waiting for the navigation between the same resources in a single foreground tab. Well, what if the browser provided an API to do this?

  <link rel="prerender" href="http://example.org/index.html">

You guessed it, that's prerendering in Chrome! Instead of just downloading a single resource, as the "prefetch" hint would have done, the "prerender" attribute indicates to Chrome that it should, well, prerender the page in a hidden tab, along with all of its subresources. The hidden tab itself is invisible to the user, but when the user triggers the navigation, the tab is swapped in from the background for an "instant experience".

Curious to try it out? You can visit prerender-test.appspot.com for a hands on demo, and see the history and status of the prerendered pages for your profile by visiting: chrome://net-internals/#prerender

As you would expect, rendering an entire page in a hidden tab can consume a lot of resources, both CPU and network, and hence should only be used in cases where we have high confidence that the hidden tab will be used! For example, when you are using the Omnibox, a prerender may be triggered for the a high confidence suggestion. Similarly, Google Search sometimes adds the prerender hint to its markup if it estimates that the first search result is a highly confidence destination (aka, Google Instant Pages):

Note that you can also add prerender hints to your own site! However, before you do, note that prerendering has a number of restrictions and limitations, which you should keep in mind:

At most one prerender tab is allowed across all processes
HTTPS and pages with HTTP authentication are not allowed
Prerendering is abandoned if the requested resource, or any of its subresources need to make a non-idempotent request (only GET requests allowed)
All resources are fetched with lowest network priority
The page is rendered with lowest CPU priority
The page is abandoned if memory requirements exceed 100MB
Plugin initialization is deferred, and pre-rendering is abandoned if an HTML5 media element is present

In other words, prerendering is not guaranteed to happen and only applies to pages where it is safe. Additionally, since JavaScript and other logic may be executed within the hidden page, it is best practice to leverage the Page Visibility API to detect if the page is visible - which is something you should be doing anyway!

Chrome gets faster as you use it # ↑

Needless to say, Chrome's network stack is much more than a simple socket manager. Our whirlwind tour covered the many levels of potential optimizations that are performed transparently in the background, as you navigate the web. The more Chrome learns about the topology of the web and your browsing patterns, the better it can do its job. Almost like magic, Chrome gets faster as you use it. Except, it's not magic, because now you know how it works!

Finally, it is important to note that the Chrome team continues to iterate and experiment with new ideas to improve performance - this process never stops. By the time you read this, chances are there will be new experiments and optimizations being developed, tested, or deployed. Perhaps once we reach our target destination of instant page loads (<100 ms), for each and every page, then we can take a break. Until then, there is always more work to do!

Faster Websites: Crash Course on Web Performance

2013-01-15T00:00:00-08:00

Delivering a fast and optimized user experience in the browser requires careful thinking across many layers of the stack - TCP and up. In a rather ambitious undertaking, when I got the chance to run a three hour (marathon) workshop at Devoxx 2012, I tried to do exactly that: a crash course on web performance. Even with that much time, much was left unsaid, but I'm happy with how it went - it turned out to be one of the most popular workshops.

The best part is, the video is now available online for free! The Devoxx team did an amazing job of post-processing the recording, with inline slides, full agenda navigation, and more. Check it out below. Hope you like it, and let me know if you have any feedback, comments or questions.

Web Performance Crash Course - PDF slides (9.5MB)

Life of a web request: TCP, HTTP, SPDY, Mobile and Navigation Timing

Part one covers much of the underlying networking infrastructure, which you may not think about day-to-day, but which often dictates the delivery and consequent performance of your application:

What is the impact of slow sites?
What is "fast" and how are we doing today?
Life of an HTTP request, TCP-up
Faster networks will save us, right? (Mostly) Wrong!
Bandwidth doesn't matter (much)
HTTP 1.0/1.1 and TCP performance
SPDY in 10 slides, or less
Performance of Mobile networks
Under the hood of Chrome's network stack
Navigation Timing & Real user measurement (RUM)

Life of a webpage: DOM, CSSOM, rendering, acceleration

Part two picks up from slide 79 and is focused on the architecture and the execution model of the browser:

Tokenizing, parsing, and construction of the DOM
Building the Render Tree (DOM and CSSOM)
Measuring visual & rendering performance
Hardware acceleration 101
Putting it all into practice on a synthetic example
Critical Path analysis for guardian.co.uk
Performance rules, tips and tricks to apply on your site

Deploying New Image Formats on the Web

2012-12-18T00:00:00-08:00

An average page is now over 1200kB in size and 60% of that is in images. With all the focus on performance and speed across the web performance industry, you would think that innovating on better image formats would be a top agenda item. Not so. Instead, we are living in a self-imposed world of scarcity of formats, effectively limiting ourselves to gif’s, png’s and jpeg’s.

In practice, deploying new image formats has been painful - just think back to the saga of png. But one would also hope that png was not the last. In fact, if we really want to make an impact on web performance, then image formats is the place to do it. There is absolutely no reason why we shouldn't have dozens of specialized formats, each tailored for a specific case and type of image. But before we get there, we need to iron out some kinks...

Deploying new "Magic Image Format"

As a practical example, let's imagine we have just invented a new magic image format (mif). How do we deploy it? As a recent bug on W3C points out, our markup provides no facility to specify different formats for a single image. Let's assume we add such a mechanism. The syntax does not matter, I'll just make it up for the sake of an example:

<img srcset="awesome.jpeg 1x, awesome.mif 2x" alt="Use awesome MIF for retina screens!">

So far, so good. The browser reads the page and decides to load the .mif file while rendering the page. The user loves our awesome image and decides to share it on their favorite social network: right click, copy URL, or simply drags the image into a bookmarklet or extension. At this point, we have a problem. Our user's friends may use a different browser, which may not support .mif files. Instead of an awesome image, they see a broken asset.

Why did this occur? When presented with all the available image options, the browser was able to negotiate the format for the user, but then the user made the singular choice for all of his friends, and in the process broke it for them. Client-driven negotiation breaks down the moment the resource leaves the page where all of the representations are available.

Now, in theory, the browsers could address this by ensuring that every time you grab the asset URL, either via drag and drop, right click, or even JavaScript interaction, then a "safe" URL is returned. However, in the long run, I don't think this is the right solution.

Humans don't scale

Quick, what is the format for the image on the left? Trick question. The file is saved as a .png on my origin server, but depending on your browser, it's either coming up as a .jpeg, or a .webp as you're viewing this page. PageSpeed Service proxy tested the various formats, and decided to re-encode the image to achieve better compression (see settings).

In fact, we already know, based on empirical results, that we as humans are terrible at optimizing images: we forget to resize them, we pick the wrong formats, and its tedious work. You would think that with three image formats to choose from, such problems would not exist. Not so. I've long stopped worrying about hand-optimizing images. Computers are much better at this task than we are, and they also don't mind the boring work.

The format doesn't actually matter

For the sake of an argument, let's say we do hand-optimize each image asset. Next, enumerate each variant:

  <img srcset="path/awesome.jpeg, path/awesome.png, path/awesome.webp,
                      path/awesome.svg, path/awesome.mif, path/awesome.mif2"
          alt="How many times do I need to repeat myself?">

That is silly, boring, and a waste of bytes in the markup. We'll need automation tools just to simplify the task of generating the repeated boilerplate - not unlike the existing CSS prefix woes. So, with that, the extension of the image file doesn't actually matter - we can do without it.

Let's imagine that we instead invent an abstract .img format, which acts as a stand-in for the optimal format. What is optimal? It is a function of the image contents and the user agent preference and capabilities at the time of the request. Who performs the optimization? The server, of course. Given a single source image, it is able to re-encode, recompress, resize, strip unnecessary metadata, ..., and deliver the optimal format.

In a world with dozens of image formats, the human solution does not scale - read, markup does not scale. Whereas computers are fantastic at doing exactly the kind of optimization work required to solve the problem.

Content-type negotiation

Good news, HTTP 1.1 already anticipated all of the mechanics to make this work with server-driven negotiation:

User agent indicates which file types it supports or is willing to accept through Accept request header
Server selects the format and indicates the returned type through Content-Type response header

Note that the extension on the image file in the URL does not matter. The same URL can return a different representation based on the negotiated client-server preferences, and the browser then uses the Content-Type to properly interpret the response. No need for extra markup, no need to hand-tune each image. Further, users don't care which format is negotiated, as long as the image works, and as long as it is delivered quickly.

Outlines of the solution

If content negotiation is already here, then why are we even having this discussion? On paper we have all we need, in practice, there are a few implementation issues to resolve. First, let's look at the Accept headers sent by modern browsers when making a request for an image resource:

Chrome	/
Safari	/
Firefox	image/png,image/;q=0.8,/*;q=0.5
Internet Explorer	image/png,image/svg+xml,image/;q=0.8, /*;q=0.5
Opera	text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, /;q=0.1

Chrome and Safari headers are effectively useless - we accept everything! Firefox and IE aren't doing much better. Opera is the only one explicitly enumerating the supported filetypes, which is the behavior we want, albeit it also adds some unnecessary types at the front. If we want server-driven negotiation to work, then the first task is to get the browsers to send a useful Accept header - a header which actually enumerates the supported types.

However, fixing the Accept header is only half the problem. The fact that the same URL may have multiple representations means that all the intermediate caches must have a way to differentiate the various responses. Thankfully, HTTP 1.1 has a mechanism for that as well, the Vary header.

  (client)  >  Accept: image/jpeg, image/png, image/mif
  (server) >  Content-Type: image/mif
              >  Vary: Accept
              >  (object)

The server indicates to upstream clients that the resource should be varied based on the value of client's Accept header by returning Vary: Accept. In the example above, given the choice of three formats, the server chose mif as the optimal one. Any upstream cache can safely cache and serve the mif object to any user agent which provides the the same Accept header. If another user agent sends a different header value, for example without image/mif, then a different format will be served and cached. To the user, this negotiation is transparent.

But, but, but...

But this puts more header bytes on the wire! Yes, it does. If we explicitly enumerate all image types, then the header is 50-100 bytes in the upstream. Half of the requests (~40) on an average page are image requests, which means ~2~4kB in total. In the downlink, these images cost us ~600kB. Assuming we can get 10-30% better compression - a realistic number for WebP, as we'll see below - this translates to 60-180kB in savings, and a 30-45x return on investment. And ~~I hope~~ I know we can do even better in the future.

Further, for HTTP 2.0, we will have header compression, which will amortize the cost of sending the image header down to a single transfer of 50-100 bytes, instead of the ~2~4kB in current overhead. As for HTTP 1.1, we can be smart and provide a site controlled opt-in mechanism, such that only sites which support the new negotiation will get the updated header from the browser - the mechanics of this deserves its own separate discussion.

But this would fragment the cache! Small changes in the Accept header could potentially create duplicate entries in the cache. This is nothing new, and the spec documents ways to normalize the headers: downcase, order does not matter, etc. Also, "Key" proposal is specifically designed to resolve this issue in a generic, cache-friendly way.

But cache support for Vary is missing! Turns out, most CDN's will simply not cache anything with Vary: Accept. That's a sad state of affairs and should be considered a bug. The good news is, this is not rocket science, rather it is a question of business incentives. Better image optimization translates to fewer bytes on the wire and better performance - this is aligned with what every CDN is trying to sell you. If the client support is there, then support for Vary: Accept is a competitive edge for every cache and CDN provider.

But old clients will break with Vary: Accept! Support for Vary has been spotty in older clients: IE6 won't cache any asset with Vary, IE7 will cache but makes a conditional request, and so on. One practical approach is to make this a forward looking optimization where only newer clients would trigger the new behavior.

But now I need new server software to optimize the images! Yes, you will. This is once again a question of incentives for hosting providers and CDN's. In fact, many CDN's already perform image optimization at the edge. Similarly, open-source projects like mod_pagespeed and ngx_pagespeed are drop-in modules, which will do all the work required to make this work.

But this means more load on the server! Dynamic image optimization doesn't come for free, but your time is more valuable. The server can optimize the asset, cache it, and be done with it. There is no global shortage of CPU cycles, and once there is an incentive, these image optimization workflows will be tuned into oblivion.

Hands on example with WebP

WebP is a new image format that provides lossless and lossy compression for images on the web. WebP lossless images are 26% smaller in size compared to PNGs. WebP lossy images are 25-34% smaller in size compared to JPEG images at equivalent SSIM index. WebP supports lossless transparency (also known as alpha channel) with just 22% additional bytes. Transparency is also supported with lossy compression and typically provides 3x smaller file sizes compared to PNG when lossy compression is acceptable for the red/green/blue color channels.

25-35% savings over PNG and JPEG, and up to 60%+ for PNG’s with an alpha (transparency) channel. That's hundreds of kilobytes of savings on most every page. Something worth fighting for.

Both Chrome and Opera support WebP, as do some optimization proxies such as mod_pagespeed, PageSpeed Service, Torbit, and few others. However, because of the lacking context in existing Accept headers, each is forced to mark .webp resources with a Cache-Control: private header. This effectively forces every request to be routed to the optimizing proxy, which then performs user agent detection and serves the appropriate content type.

As many have pointed out, this method does not scale: every request is routed to the origin server, and marking the resource as private bypasses all the intermediate caches. This alone is enough of a reason for why we haven’t seen WebP, and other experimental formats, get any significant adoption on the modern web.

The "action plan"

Step one, we need to fix and normalize the Accept headers. Opera's header is the closest to what we want, albeit we can restrict the content-types for image assets when requesting an image resource. Perhaps something like:

  Accept: image/webp, image/png, image/jpeg, image/gif, image/svg+xml, image/bitmap

With that in place, there is an incentive for the CDNs, proxies, and servers to perform the negotiation to deliver the optimal image format. Finally, once the server can choose the format, the upstream caches have an incentive to make Vary work. In the end, we shave off hundreds of kilobytes of image data, we automate the menial task of selecting the optimal image formats, and we have a future-proof negotiation mechanism which can scale to dozens of image formats. Everyone wins... until we offset the win by embedding more cat pictures on our pages...

Web Performance Anomaly Detection with Google Analytics

2012-11-30T00:00:00-08:00

Step one, monitor all the things. Step two, dedicate 90% of your analytics time and resources to analyzing data, deriving insights, and iterating on what metrics are being monitored and are being optimized. However, there is one small problem. Chances are, the amount of data produced by the instrumentation outpaces your ability to analyze, monitor, and correlate all the variations of the variables at play.

This is where an anomaly detection algorithm, backed by a good statistical engine with access to the data can prove invaluable: it does not need to be perfect, but it should be able to alert you to significant outliers in the data. With the alert in hand, or in your mailbox, you can dig in and determine if more investigation is required.

Google Analytics Intelligence Events

Good news, if you are using Google Analytics, then you already have a powerful anomaly detection engine at your disposal: Intelligence Events. Best of all, it can leverage all of your existing data, configured segments, and other customizations. And the price is right as well - it's free.

Analytics monitors your website's traffic to detect significant statistical variations, and then automatically generates alerts, or Intelligence Events, when those variations occur. Taking a closer look at these anomalies can provide insights you might have otherwise missed, for example, a spike in traffic from a particular city or referring site.

In fact, with a little bit of work and customization, Intelligence Events can be easily configured to help you monitor the performance of your site! Visitors from India seeing a sudden spike in page load times? Now you have an automated tool which will help you spot the problem.

Even better, the sample report above correlated the likely contributors to the generated alert and identified visitors from Chennai, India as experiencing a significant increase in their page loading times. With this information in hand, you can dig deeper to identify root cause.

Web performance anomaly detection

Google Analytics samples page load time performance data for browsers that support the W3C Navigation Timing API's, which includes: redirect and DNS times, TCP establishment, server response times, as well as DOM-level metrics such as the onload time. There are over half a dozen metrics in total, each recorded from a real user accessing your site - in other words, this is Real User Measurement (RUM), not synthetic data.

If you're not familiar with the Site Speed reports, then that's a good place to start - check out this GDL episode for an in-depth look. However, we're going to go a level deeper: each of the Navigation Timing metrics can be monitored with Intelligence Events! All you need to do is create a custom alert and define a few threshold criteria which will trigger it in the future.

A few performance alert ideas for your site:

Track DNS resolve times across the world, or in specific regions
Track server response times across all visitors, or customize for different versions of the site
Track onload time to detect misbehaving CSS, scripts, and other resources

Powertip: Use Advanced Segments!

Setting up an alert for a specific variable, within a global context, is a good place to start. However, if you have not created a custom advanced segment in Google Analytics, then you've only scratched the surface of what's possible. Need to monitor the page load time (PLT), or DNS times for all mobile visitors in Asia, or perhaps Tokyo specifically? No problem, just create a new custom segment:

You can apply the segment on any report in Google Analytics, and once created, you can also select it when setting up the Intelligence Alert! If there are specific markets, or types of users or traffic that you're concerned about, then creating an advanced segment will allow you to tailor the alerts as well.

Measure, Optimize, Iterate

Anomaly detection is a tool, and a powerful one at that. However, it is still up to you to define, and iteratively improve the segments and the thresholds to tailor the alerts to your application. Expect false positives, but also expect to be alerted to issues that you would never otherwise have caught in the torrent of the monitoring data.

Wait, Chrome DevTools could do THAT?

2012-11-14T00:00:00-08:00

Your browser is one of the most and best instrumented development platforms - you may just not realize it yet. Of course, you can inspect the source, walk the DOM, fiddle with the CSS, and evaluate and debug your JavaScript, but there is so much more! Granted, there is always room for improvement, but the exciting part is that much of this functionality is also built with the same technology we build our sites: HTML, CSS, JavaScript, and WebSockets. This means we can extend it, leverage it for use cases outside the browser, and much more.

To highlight some of these features, I gave a presentation at Velocity EU a few weeks back: "Wait, Chrome DevTools could do THAT?". To my surprise, it became a runaway hit, aggregating over 100K unique visitors since published, and generating a lot of requests for the video of the talk... See below!

Wait, Chrome Dev Tools could do THAT?

This episode provides a whirlwind tour of how to analyze network performance, rendering and layout pipeline, as well as detecting memory leaks in your JavaScript, and using audits and extensions to build faster apps!

Extending Chrome DevTools for fun and profit...

This episode takes a deep(er) dive into the inner workings of Chrome itself: the debugging protocol, and Audit and Panel extension API's in DevTools, a quick tour of the Chrome's network stack, and chrome://tracing features!

I encourage you to try these tools for yourself by following the links and examples in the presentation: see the slides. Finaly, if you have any other tips, suggestions, or favorite features I've omitted - let me know!

Simple SPDY and NPN Negotiation with HAProxy

2012-10-31T00:00:00-07:00

SPDY is an experimental protocol developed at Google, designed to reduce the latency of web pages. Specifically, its goal is to address the limitations of HTTP 1.1 and to remove existing bottlenecks: head of line blocking, inefficient use of underlying TCP connections, and header bloat amongst others. However, while all of this sounds great in writing, deploying a new protocol on the web, in practice, is fraught with difficulty.

Specifically, the many layers of HTTP routers, caches, and custom appliances will often behave in unpredictable ways when presented with an alternate protocol, leading to randomly dropped connections and frustrated users. To address this, SPDY is delivered via SSL: the end-to-end encrypted tunnel allows the client and the server to exchange SPDY frames without intervention by intermediate nodes. It's important to note that SPDY does not require SSL, but in practice, SSL is a pragmatic choice to get to a working solution.

Next Protocol Negotiation (NPN)

But how does the client and server know to use SPDY once the SSL tunnel is opened? This is where Next Protocol Negotiation (NPN) enters the picture. NPN is an SSL extension which allows the client and the server to negotiate the application protocol as part of the SSL handshake.

NPN eliminates the extra roundtrip to negotiate the application protocol - contrast this to the current WebSocket handshake, which imposes another roundtrip of latency on top of the SSL negotiation. If you're counting, that is one RTT for the TCP handshake, two for SSL negotiation, and one for WebSocket upgrade - four RTT's before any application data can be exchanged!

Support for NPN was added in OpenSSL (1.0.0d+), NSS, and TLSLite. Chrome, Firefox, and Opera support SPDY (and hence NPN also), as well as many of the popular servers: Apache, nginx, Jetty, and node.js amongst others. That said, the requirement for NPN support has nonetheless been a barrier to wider adoption. That is, until HAProxy decided to make everyone's life a lot easier! Let's take a look at a hands on example.

Building HAProxy with NPN support

Early SSL support landed in HAProxy development builds in early September and has been rapidly improving ever since. A notable and a recent addition is, you guessed it, support for NPN negotiation! To make it work, download the latest development tarball, or do a checkout from git and build the binary against a recent version of OpenSSL:

$> wget http://www.openssl.org/source/openssl-1.0.1c.tar.gz
$> tar -zxvf openssl* && cd openssl*
$> ./configure --prefix=/tmp/openssl --openssldir=/tmp/openssl darwin64-x86_64-cc
$> make && make install

$> git clone http://git.1wt.eu/git/haproxy.git/ && cd haproxy
$> make TARGET=generic USE_OPENSSL=1 ADDINC=-I/tmp/openssl/include ADDLIB=-L/tmp/openssl/lib -lssl
$> ./haproxy -vv
  # Built with OpenSSL version : OpenSSL 1.0.1c 10 May 2012
  # OpenSSL library supports TLS extensions : yes

The above instructions should build the latest version of HAProxy on OSX. If you already have OpenSSL 1.0.0d+, then you can skip the first step, as well as the extra ADDINC and ADDLIB directives. With that, we're almost ready to handle SPDY alongside our regular HTTP and HTTPS traffic!

Configuring HAProxy for HTTP, HTTPS, and SPDY

What we want to do is to configure our HAProxy as an SSL termination proxy. Meaning, HAProxy will be the one serving our SSL certificate back to the client, and all traffic forwarded to our internal servers will flow unencrypted. This also means that HAProxy will need to handle the NPN handshake. In fact, ideally, it should handle and route all types of traffic: HTTP, HTTPS, and SPDY.

defaults
  log 127.0.0.1 local0

# accept connections on port 80, forward requests to "http_cluster"
frontend http
  mode http
  bind :80
  default_backend http_cluster

# accept connections on port 443 (SSL)
# - forward SPDY connections to spdy_cluster
# - forward regular HTTPS connections to http_cluster
# - ha.pem should be a concatenated file: certificate first then the key
frontend secure
  mode tcp

  # advertise spdy/2 support via NPN
  bind :443 ssl crt ./certs/ha.pem npn spdy/2
  use_backend spdy_cluster if { ssl_npn -i spdy/2 }

  default_backend http_cluster

# SPDY server running on port 10000
backend spdy_cluster
  server srv01 127.0.0.1:10000

# HTTP server running on port 8000
backend http_cluster
  mode http
  server srv01 127.0.0.1:8000

Simple as that. HAProxy accepts the SSL connection, performs the SSL handshake and advertises spdy/2 support if client supports NPN. If the client selects spdy/2 then the request is routed to the SPDY server. Otherwise, the HTTPS request is forwarded to the HTTP cluster. The important bit is that in both cases, the forwarded traffic is no longer encrypted - the SPDY server does not need to perform the SSL handshake or worry about NPN, instead it simply has to parse and return the raw SPDY frames back to the proxy.

NPN is not just for SPDY

To see SPDY in action, you can checkout and run the demo SPDY server written in Ruby behind HAProxy. With native NPN support in HAProxy, adding SPDY support has never been easier - you are literally just a few rules away from intelligently routing incoming SPDY requests alongside the rest of your traffic. No need to downgrade your SPDY connection to HTTP, punch holes in your infrastructure to expose NPN capable serves, or deploy parallel infrastructure.

Best of all, while the above example is specific to SPDY, the NPN mechanism works for any new, or existing protocol. As an example, we can eliminate the extra roundtrip in WebSocket negotiation with NPN, or we can deploy other, new and experimental protocols without imposing any additional latency penalties.