Reading Jeff Hodges’ excellent post, containing “rubber meets the road” advice for budding engineers in distributed computing projects, caused me to reflect on the rise of ad hoc engineering over the certainty of science.
When I took my first college computer science course, the emphasis was on the science part of computer science, and “writing code” was primarily proof that you understood the science. As dissatisfying as that seemed at the time, my theoretical start later felt like a competitive advantage in a world dominated by so-called hackers: often brilliant, rarely disciplined.
In more recent years, the ability to quickly hack together a product, often by combining existing open source technologies in some novel combination has become the foundational technology for companies such as Twitter, Instagram, and GitHub. My impression, though, is these are, at least initially, triumphs of software engineering more so than computer science. Perhaps the impressiveness of these engineering accomplishments lead some to think that most computer software problems are software engineering problems, as opposed to computer science problems?
Much of Jeff’s post is a warning to an unwary hacker plowing into engineering of distributed systems. Simplistic ideas around replication and transaction coordination are the ticket to a rabbit hole from which there is no escape. Put simply: you can’t debug your way out of knowing the math.
And there is a mention of math, although it’s in a section titled “Coordination is very hard” and goes on to recommend you avoid it whenever possible. This is, of course, my excuse to mention our patented, WAN-capable Paxos coordination implementation called DConE. DConE currently powers our SCM related CVS and Subversion MultiSite products, and will soon to come to bear on other important industry needs. In contrast to software engineering driven technology, WANdisco’s core technology is clearly and firmly rooted in computer science.
You’ll not catch me saying engineering or science is better. Different types of problems and opportunities will advise different emphasis on one side or the other. But in the world of distributed computing, you’d better come armed with deep knowledge of the science.