Agile RSS Aggregator in Ruby

By Ilya Grigorik on March 22, 2007

A liberal RSS aggregator in 26 lines of Ruby - may be hard to believe, but it's true. On top of that, it is also capable of serving static files, has a templating engine, and accepts an arbitrary number and types of RSS feeds - talk about squeezing functionality out of every line!

Leveraging Ruby libraries

Knowing the right libraries is a godsend in any language, but when this knowledge is combined with Ruby, all kinds of spectacular things begin to happen. In this case, I relied on four wonderful gems: open-uri, feed-normalizer, erubis, and mongrel. Putting all four together, we get our RSS aggregator:

require 'open-uri'
require 'feed-normalizer'
require 'erubis'
require 'mongrel'

class RSSHandler < Mongrel::HttpHandler
  def process(request, response)
    response.start(200) do |head,out|
      head["Content-Type"] = "text/html"

      stories = []
      File.open('feeds.txt', 'r').each_line { |f|
        feed = FeedNormalizer::FeedNormalizer.parse open(f.strip)
        stories.push(*feed.entries)
      }

      eruby = Erubis::Eruby.new(File.read('news.eruby'))
      out.write(eruby.result(binding()))
    end
  end
end

h = Mongrel::HttpServer.new("0.0.0.0", "80")
h.register("/", RSSHandler.new)
h.register("/files", Mongrel::DirHandler.new("files/"))
h.run.join

Demystifying the magic

Starting from the top, let's take a look at how each library contributes to the final product. First, Open-URI extends the open() method call in Ruby to support HTTP/HTTPS requests, which we will use to retrieve the feed. Next, Feed-Normalizer acts as a wrapper for several RSS/Atom parsers, and returns a unified object graph. For a templating engine, we will use Erubis, which is a fast, lightweight implementation of Eruby. And finally, Mongrel will be our HTTP library/server, and it will serve our live aggregated feed back to the user.

Open-URI and Feed-Normalizer

Both libraries go to work right in the middle of the RSSHandler class. First, we read the feeds.txt file (every line represents a feed url) and then pass the url to open(), which is handled by Open-URI. Next, we return the retrieved feed to the FeedNormalizer parser. The end result, a unified object tree for RSS and Atom feeds, all in one line!

Erubis and Mongrel

Once we have the stories, Erubis comes into the fold to take care of the templating part. I created a small html file, which includes the following code:

<% for item in stories %>
  <div class="section details">
  <h3><a href="<%= item.urls.first %>"><%= item.title %></a></h3>
     <p style="color:#444;font-size:90%"><%= item.content %></p>
  </div>
<% end %>

Rails developers should immediately recognize Erb syntax. In the aggregator, we defined a 'stories' array, which we bind to the template and evaluate with Erubis. Next, to dynamically display the aggregated feed, our RSSHandler class inherits from Mongrel's HTTPHandler class and serves the final result from Erubis back to the client. Finally, for the bonus points, we also mount a directory handler in Mongrel just so we can serve static files - a stylesheet, in this case.

That's all there is to it. How can you not love Ruby after that!

Ilya Grigorik is a web ecosystem engineer, author of High Performance Browser Networking (O'Reilly), and Principal Engineer at Shopify — follow on Twitter.