Henry Ford & Event Driven Architecture
Talking about architecture, as with most things in the computer industry, often requires deciphering a soup of three or four letter acronyms (XML, API, SOAP), all of which can often be described in much simpler terms. Working on a presentation on "Event Driven Architecture" for MeshU, I was struck how we've managed to obscure such a simple and familiar concept into something that few developers can understand: it's an assembly line!
As an industry, we've come a long way in the past decade. Not so long ago, everyone rolled their own flat-file databases, nowadays the challenge is with storing and accessing terabytes of data. However, a lot of this innovation is often a process of reinventing the wheel. It may not seem so trendy, but looking back exactly a century ago is a good place to start: the manufacturing & assembly line revolution.
Henry Ford: Beyond Horizontal Scalability
Horizontal scalability is all the rage, that is, if you can achieve it. If you've managed to abstract your database layer, then just keep adding servers, hide them behind a collection of proxy servers, and you're in the money. It works, up to a point.
What's remarkable is how similar this pattern is to the way a car was manufactured in the early 1900's. A team of two or three men would work a car, start to finish, and would take anywhere from a day or two to finish the order. Need more capacity, hire more teams! Life is great.
Then Henry Ford came along and changed the game. The simple insight that the efficiency can be increased by an order of magnitude by creating a workflow, as opposed to optimizing the start-to-finish process, is actually quite remarkable. Ford invented the assembly line, we're calling it "Staged Event Driven Architecture (SEDA)". End result, ninety three seconds to manufacture a car vs one to two days, and over one million cars produced every year.
Event Driven Architecture: The steps
First and foremost, stop thinking about time to complete. Think workflow instead. There is no reason to hold up the front of the line if someone else can do the work later. By deferring the work you cut down your response time, which means snappy user experience, and it also means that you're able to take advantage of the latest industry buzzwords: elastic computing, and real-time web. Keywords to explore: AMQP, XMPP, cloud computing. Or, take a look at the slides from my MeshU presentation:
Cloud Computing: Assembly Line 2.0
It's tacky, but cloud computing is "assembly line 2.0". Even given all the industry hype, it is hard to underestimate the impact that cloud computing is going to have on our industry. All of the sudden, the cost of hiring / firing another server is zero, and the datacenter (factory) costs are swallowed by the provider - we can build and disassemble our factories in a matter of minutes! Toyota's "just in time" manufacturing is as close as it gets in the physical world, but it's only a fraction of the efficiency we can achieve in the virtual world.Having said that, we still have a thing or two to learn from the physical world.
Ilya Grigorik is a web ecosystem engineer, author of High Performance Browser Networking (O'Reilly), and Principal Engineer at Shopify — follow on Twitter.