RSCS, and Some New Thinking On Open Source

Having taken interest recently in Perl6, I have been spending some time writing libraries as a way of learning the language while making some sort of lasting contribution. Having been a casual user of Hashicorp's Consul and Vault, I decided it might be useful to create Perl6 bindings for them both, starting with Consul.

A few days into the work I threw in the towel. Even though I was limiting my imagination and focusing primarily on porting the Go client library, I found myself getting lost in the myriad options and features available. I had felt this way even as an end-user of both Consul and Vault...slightly overwhelmed by complexity that seemed unnecessary or only relevant to a tiny subset of users. I've always had a nagging feeling that if something went wrong with Consul or Vault in production, I wouldn't know what to do.

I feel like Hashicorp has adopted as a goal that their products should be useable by most of the target audience given a binary and some API documentation. Unfortunately this means these products must be meaningfully complex as it is assumed the end user will not read or extend the code. This strikes me as a closed-source software experience. To be certain, many users want this, but it is no longer for me.

Tools have been improving rapidly in the last few years: we have reasonably good programming languages with useful type systems. We also have excellent mechanisms for continuously integrating remote codebases. Given these advances, it seems the way forward is to share smaller core codebases and ask that end-users simply branch the code and extend it locally for the features they alone need, as opposed to trying to anticipate all expectations in a single upstream binary. Given a strong type system and sanitary version control practices, continuously integrating the final codebase should be a reliable and trustworthy process. A fringe benefit of this model is that it would also force end-users to read and understand code instead of working from API documentation, which is always imprecise and often inaccurate.

Scratching my itch, I decided to write something called RCSC (the Ridiculously Simple Configuration System) as a replacement for both Consul and Vault insofar as my needs and tolerances dictate. There is also a Perl6 client library. RSCS is so simple that it almost defies explanation: it is literally a single SQLite table that persists keys to values, with an HTTP daemon providing ReST/CRUD-style operations. There is no built-in Raft support. There is no DNS interface. There is no default format for describing services. There is no distinction between password storage and configuration storage. The point is to provide a small foundational service that is easy to understand at a source-code level and trivially simple to use...and if you want to add features that appeal to you directly, you fork the codebase, read it, understand it, and extend it. Given Go's reasonably good type system, it should be possible to continuously integrate upstream changes from my repository with your own local enhancements. Instead of forcing you to wade through options you do not care about, you can just write Go code to provide the functionality you care about.

Another benefit to keeping code bases pathologically simple is that it makes it easier to say no. Software developers like to accomodate and feel accomodated. So we say "yes" to requests that aren't necessary while also making requests of others...often just to demonstrate our own abilities by way of claiming requirements. We end up with monstrosities that are over-configured and over-engineered often as a result. Big tools have their place, but we should be extremely cautious about taking inspiration from their success.

last update 2017-02-19