Rails 3 Asset Pipeline & Google Closure

By Ilya Grigorik on August 16, 2011

Rails 3.1.0 is on the horizon and the new Asset Pipeline is the king of the show. If this comes as news, then make sure to check out DHH's keynote from RailsConf, and also read through the excellent edge guide. This new release brings about some of the most developer visible changes in a long time: it both makes the previously "stashed away" assets such as Javascript and CSS first class citizens in our development workflow, and also pushes us into the bold new world of Coffeescript, Sass, and many other similar tools. It's about time!

Asset Pipeline & Preprocessing

Under the hood, there are two major components: Sprockets is responsible for all the asset preprocessing, and there is an additional middleware (mounted under /assets) which manages the actual toolchain. Hence, a request to /assets/application.js will hit the middleware, which will in turn invoke Sprockets, run the minifiers and compressors, and finally return the Javascript file. Good news is, this is all basically invisible.

Let's take a closer look at the default application.js:

// javascript comment
//= require jquery
//= require mylib/stuff.js
//= require_tree otherlib/

The goal is to enable Rails to serve a single Javascript file back to our visitors. To accomplish this, we need to tell Rails which files and directories should be included in the output. With that in mind, and as you may have guessed, the //= comment is, in fact, a special directive for Sprockets to include the specified file in the processing of the asset pipeline. Just specify the files, or entire directories and the rest is taken care of. CSS handling is no different.

Google Closure + Dependency Management

Turns out, Google's Closure library has a similar, but different, built-in dependency management system:

<script src="closure-library/closure/goog/base.js"></script>
<script>goog.require('goog.dom')</script>
<script>
  var newHeader = goog.dom.createDom('h1', {}, 'Hello world!');
</script>

Closure's DOM manipulation (goog.dom) is an independent set of libraries, which are loaded by Closure at runtime when we call goog.require. Now, the dynamic loading is convenient in development mode, but it would be both slow and costly in production! To address this, Closure also ships with a separate Python script to preprocess your Javascript to generate the single asset file:

$ closure-library/closure/bin/calcdeps.py -i hello.js -p closure-library/ -o script > hello-calc.js

In other words, the net effect is the same as that provided by the built-in Asset Pipeline, but it requires manual intervention and third party tools - we can do better.

Extending Rails 3 Asset Pipeline: Closure support

The great news is, Sprockets allows us to easily modify and add new directive processors into our pipeline: there is no reason why we should be limited to //= directives only! Closure's goog.require's are effectively the same mechanism, hence we can extract these calls, load the right files, and make Closure a first class citizen in Rails! To do that, we can define a simple Closure Preprocessor:

class ClosureDependenciesProcessor < Tilt::Template
  def evaluate(context, locals, &block)
    context.require_asset 'goog/base'

    data.lines.each do |line|
      # scan the JS file for goog.require calls
      if line.match('goog\.require\s*\(\s*[\'\"]([^\)]+)[\'\"]\s*\)')
        goog, mod, sub = $1.split(".")
        sub = mod if sub.nil?

        dep = [goog, mod, sub].compact.join("/").downcase
        context.require_asset(dep)
      end
    end

    data
  end
end

With that in place, we can define a new Railtie to register the new Javascript preprocessor, package it as a gem, and we're in business! Add the gem to your Gemfile, download the closure library, and you are good to go.

Rails 3 API's

The final closure-sprockets gem is almost trivial and is even easier to setup within your own project. Having said that, this is only true because of the extensibility provided by both the Sprockets preprocessors and the Railtie integration - kudos to the Rails 3 team for putting together these great API's. Finally, thanks to Josh for hand-holding me through the process of learning about Sprockets and the new Asset Pipeline!

Ilya Grigorik is a web ecosystem engineer, author of High Performance Browser Networking (O'Reilly), and Principal Engineer at Shopify — follow on Twitter.