loose coupling for happiness
I've been building web systems, mostly in the backend, for nearly twenty years. There have been brief periods of elation, but most of the time the developers seems to be sanguine about the code they are required to work with. why? In hindsight, it seems that the issue of moving code to a better place was intractable. And why was that? I think it is because we built tightly coupled and/or monolithic systems, which by their nature are extremely difficult to deal with and usually result in costly and painful rewrites. The discipline required to maintain sanity and utility in a monolithic system is vanishingly rare and likely not worth pursuing.
If I could give one piece of advice to developers building "large" systems it would be this - be loosely coupled and you will probably be happy. Loose coupling delivers a lot of benefits:
- simple, primitive interfaces
- the ability to scale out parts of the infrastructure non-uniformly
- the ability to change the implementation of a loosely-coupled element of the system without impacting the rest
- the opportunity to distribute responsibilities on your team sensibly
But there's no free lunch. The costs of decoupling will typically be more moving parts and lower performance. But its worth it. We have a lot of options available to use for loose coupling: from protocol buffers to REST apis. Developers often talk about the pipe dream of software components that are as easy to interchange as lego bricks...I believe the first step toward this is loose coupling of services using known techniques.
where to be loosely coupled? Often it is useful to loosely couple a service that has non-uniform resource requirements, such as a process that must exploit GPUs or other special hardware. Consider services that have special time considerations that are unique in the larger system it serves. Or maybe a piece of code that is merely troublesome or deals with special security issues. There are many judgement calls to be made.
where can't you be loosely coupled? Typically inside your database. Databases are varied and opinionated these days, as are our schemas and strategies for their use. Abstracting away the core advantages of a particular data schema or storage system for the purposes of loose coupling still doesn't seem realistic or useful.
how to be loosely coupled? Widely-known utilities such as protocol buffers allow system types and structures to be visible remotely. HTTP-based systems benefit from ubiquity but sacrifice complex functionality.
REST is worth discussing further. REST/HATEOS is typically lauded as a design approach that is the zenith of hypertext design. I'm not sure I agree with full-system modeling via REST. I prefer to confine RESTful endpoints to the smallest unit of WAN coupling, corresponding closely to the smallest discrete unit of functionality from a service pursuant to the following design goals:
- caching. REST is about caching (and when not to cache). Sensible WAN-coupled services shold adhere to http caching rules to be useful and correct. In more complex applications, http caching alone is not adequate or practical. If the caching considerations of a service cannot met by http, REST may not be appropriate.
- correlation to http verbs. If an endpoint cannot be clearly explained in terms of the basic verbs, it is likely too complex to be exposed via REST.
When services cannot be practically exposed via RESTful endpoints, other techniques which may be beneficial are structured input formats (e.g. JSON formatted request bodies) POSTed to a service, or query languages. While more complex to implement, query languages can often be easy for clients to use and provide for advanced optimization on the server that are not possible with poorly designed RESTful endpoints.
The design of the software itself is often as important as the techniques used to couple. A loosely-coupled service is focused on processing and transmitting data. Therefore it is useful to borrow techniques from functional programming that emphasize dataflow and safe handling of state, particularly IO interactions †.
Beneath implementation is the foundational realm of interfaces, where you define the contracts of your services independently and as part of the full system. This is where you want to spend your time and effort, and where object-orientation can actually be useful. I believe that the correct design and architecture of interfaces as opposed to implementation is one of the unsolved mysteries of systems design and worthy of further exploration.
† It is important not to trivialize the impact of IO. Its not a simple matter of isolating functions that deal directly with IO activity. This might have been the case at one point, but with datasets becoming larger, and operations becoming longer in duration, its inevitable that most of our apis will be distinguished as either lazy (data treated as a segmented or on-demand stream) or strict (you wait for everything to be ready and then get it all). Laziness can have a cost in memory and performance, while allowing larger and more interesting operations to be performed at all. Strict functions will be easier to reason about, but may inevitably fail at scale. There are implications for loose coupling: mixing lazy and strict operations without thought will result in misery.
last update 2013-10-12