Automating DPR switching with Client-Hints

By on August 29, 2013

Note: Updated Oct 30, 2013 to reflect new Client Hints draft.

High-density displays pack multiple physical pixels per CSS pixel, and the ratio between the two is known as the device pixel ratio (DPR). To deliver the best visual quality, a HiDPI image that matches the DPR value of the device should be served to the client. The question is, how?

The answer to date has been to use CSS media queries, but that's both cumbersome and insufficient: it's a CSS only solution, img tag is more or less useless, etc. In response, we now have a number of proposals on the table: picture element, srcset, and the lesser known image-set. Much ink has already been spilled on the pros and cons of each, but the new and notable development is that WebKit has recently landed an implementation of srcset.

srcset is good enough (tm)

Both srcset and picture tag are much more ambitious and go well beyond DPR-switching, but it's important to note that the current WebKit implementation tackles a subset of the larger spec - specifically, just the DPR switching. You can look at this in two ways: as a failure because we haven't come up with "the solution", or as a necessary and well overdue positive first step.

I happen to be in the latter group. In fact, I think it's a big positive step (it's good enough), and I hope other browsers will follow. Having said that, I do have some reservations as well. Let's take a look at what's on the table:

<img src="img.jpg" srcset="img-3.0.jpg 3x,   img-2.0.jpg 2x,
                                       img-1.5.jpg 1.5x, img-1.0.jpg 1x"  />
#someid {
  background-image: image-set(
                          url(img-3.0.jpg) 3x,   url(img-2.0.jpg) 2x,
                          url(img-2.0.jpg) 1.5x, url(img-1.0.jpg) 1x)
}

Call me crazy, but that seems silly. I know web developers relish writing repeated boilerplate code, and better, inflating our markup with complex-looking syntax because it helps our job security, but... </sarcasm>, perhaps there is an easier way? How about:

<img src="img.jpg">
#someid { background-image: url(img.jpg) }

Yes, the old-fashioned img tag and url() could do just fine while delivering all the same benefits as the example above, but without the bloat, and even better, without requiring any changes to how we write our code today. The browser can do all the work on our behalf and HTTP already provides the mechanisms to handle this.

Client-Hints and DPR negotiation

Client Hints can be used as input to proactive content negotiation; just as the Accept header allowed clients to indicate what formats they prefer, Client Hints allow clients to indicate a list of device and agent specific preferences.

You can find the full spec on the IETF tracker, but the above makes it sound way more complicated than it really is. In a nutshell, just as we can now use the Accept header to determine which image format the client supports, the CH-DPR (aka Client-Hints DPR) header communicates the DPR resolution of the device to the server, and the server provides the appropriate asset. In HTTP speak, this work as follows:

(request)
GET /img.jpg HTTP/1.1
User-Agent: Awesome Browser
Accept: image/webp, image/jpg
CH-DPR: 2.0

(response)
HTTP/1.1 200 OK
Server: Awesome Server
Content-Type: image/jpg
Content-Length: 124523
Vary: CH-DPR
DPR: 2.0

(image data)

The client sends the CH-DPR: 2.0 hint to the server, and the server can choose to use this information to serve the approtiate image. If the server opts-out, or ignores the the CH-DPR header, then it can serve the default image - no harm done. But if it does elect to use the CH-DPR hint, then it also appends the Vary: CH-DPR header to indicate to upstream caches that the value of CH-DPR header should be used as part of the cache key. Nothing new, moving along.

Hands on with Client-Hints and Nginx

It doesn't take much to implement a "DPR aware" server. For the sake of an example, lets do just that with Nginx:

http {
  # - capture the DPR resolution from CH-DPR header
  # - if no CH-DPR header is present, default to 1.0
  # - round DPR ranges to neares available number asset (1.0, 1.5, 2.0)
  map $http_ch_dpr $dpr {
    default "1.0";
    ~1\.[01234] "1.0";
    ~1\.[56789] "1.5";
    ~2\.[0123456789] "2.0";
  }

  server {
    # - check for appropriate DPR asset
    # - if appropriate DPR is not available, fallback to original
    location ~* /images/(?<name>.*)\.(?<ext>.*)$ {
      add_header DPR $dpr;
      add_header Vary CH-DPR;
      try_files /images/$name-$dpr.$ext $uri =404;
    }

    # ... regular nginx configuration
  }
}

Seven lines of code, that's all it takes. First, the map directive captures the DPR value out of the CH-DPR header, and set is to "1.0" if missing. Then, we create a new location directive which intercepts request to /images/ folder and rewrite the request by looking up the appropriate file on disk. If no such file is found, we try the original file, and finally 404 if that is missing as well. Let's test it out:

$> curl -s  http://localhost:8080/images/awesome.jpg | wc -c
  252680
$> curl -s -H"CH-DPR: 1.5" http://localhost:8080/images/awesome.jpg | wc -c
  381135
$> curl -s -H"CH-DPR: 2.0" http://localhost:8080/images/awesome.jpg | wc -c
  710175

The first request does not contain the CH-DPR header and the server returns the default (1.0) asset. In the second request, we add the CH-DPR header with DPR value set to "1.5" and a different image is served, which you can see by the byte-count. Magic! Except, it's not. Just vanilla HTTP doing exactly what it was designed for.

How did Nginx produce the right DPR asset? In this case, I simply pre-generated the right assets and placed them on disk: awesome-1.0.jpg, awesome-1.5.jpg, and awesome-2.0.jpg. It doesn't take much to add a build step using a tool like grunt to automate this part of your workflow. Alternatively, the request could have been proxied to an image service (local or third-party), which can create these assets dynamically. Point being, this is automation at work. I like automation; I'm lazy; I have more important problems to solve than to repeat boilerplate markup.

Update, Oct 30, 2013: check out sample reference CH-DPR server on GitHub.

But, but, but...

But this puts more header bytes on the wire! Yes, it does. Repeated boilerplate markup puts a lot more bytes on the wire. Granted, the HTTP 1.x header bytes are uncompressed, but even with that the CH-DPR header will add a whopping 11 bytes. Once you repeat all of the markup boilerplate for every DPR breakpoint, for every image, CH-DPR will come out ahead by a good margin. Client-Hints will save bytes.

But... this will fragment the cache, and Vary support is "missing", and old clients don't work well with Vary? I've tackled all of these previously when talking about Accept negotiation. Since then, we've had multiple CDN's successfully deploy Accept negotiation for WebP delivery, and we now have big services (including at Google) relying on Accept negotiation.

But we can automate boilerplate generation! Yes, we could. Extra build steps could parse the HTML and CSS and rewrite it to inject the appropriate markup. I'll let you be the judge whether that's a "better solution". Having said that, I'm not opposed to implementing srcset, I just happen to think that Client-Hints addresses the problem in a more elegant way.

But my server doesn't do X! Get a better server. Yes, I'm serious. That's a much cheaper solution in the long run - your time is more valuable. Further, if Client-Hints is adopted, you can bet that there will be dozens of plugins and modules available in no time flat to automate the entire problem. As we saw above, it doesn't take much.

Simplifying the "responsive images" discussion

DPR switching is only one part of the larger "responsive images" discussion: we also have resolution switching, art-direction, and an increasing chorus of people asking for "pixel perfect" images (i.e. images scaled to exact size in the viewport to avoid client resizing). These are independent problems and we should treat them as such.

In fact, I would argue that the only solution that requires explicit markup is the art-direction case - by definition, that's up to the designer to dictate. The rest is just boilerplate that can and should be automated. I don't want to think about this stuff, it should just work. HTTP can help.

Ilya GrigorikIlya Grigorik is a web ecosystem engineer, author of High Performance Browser Networking (O'Reilly), and Principal Engineer at Shopify — follow on Twitter.