Environment-Oriented Programming

Over the past few years I've noticed a recurring theme in large projects I have worked on: the need to formalize state environments. Concretely, this has been the formalization of testing, development, staging and production environments and their different requirements.

Managing state in these different environments has typically been hacked in or externalized somehow, but eventually these concerns seem to dominate the exercise of large-scale development and also just be the biggest pain point.

As a result, I've taken to having a fixed set of environments I develop for and a list of technologies that can be relevant to satisfy the needs of each environment.

Environment Types And Solutions

• Unit - Used for unit testing and the most minimal vetting of design choices. All state should be transient and require no annoying build-up or teardown phases. I still don't like mocks for this use-case and fortunately there are often good choices for fully-functional, yet fully-transient state mechanisms which make mocks unnecessary. At this level, some tools I find useful include:
  • SQLite with the in-memory database option. I typically work with Postgres but alas, I don't see a similar transient option. Since most of my schemas are fairly simplistic, portability issues aren't that much of a concern as long as it is understood that using a different database for unit testing means that further testing with a production-grade database is still necessary.
  • Afero with the in-memory filesystem. Afero is just awesome. I really wish the ability to abstract filesystem operations from the underlying implementation had been part of Go from day one.
  • Miniredis. Miniredis lets you start up a fake (yet basically fully functional) Redis "server" and use it for the course of unit testing, and then just dispose of it.
  • Go's Test Http server. This is already well known and is an essential part of many developer's test infrastructure.
• Development/PR - Used to demonstrate applicability of changes to a real system, but still with enough support for transience that build-up and teardown can be automated. At this level, typical choices include:
  • CI Systems. I'm not a fan of most CI systems in use today as they seem to rely on kludgy YAML configuration and tend to be inflexible...but I understand the need to share work with others and CI systems are the best way to do that.
  • Containers. At this level it is important to normalize execution contexts and containers are obviously the way to do this for the time being.
  • Docker Compose. I love Compose. Compose allows me to synthesize an entire topology in a transient fashion, even if some of the state mechanisms employed under the hood have no notion of transience.
• Staging and Production. These are harder to discuss because there is so much variance in industry. Some are on AWS, GCP or Azure. Others are on-prem. Others are trying to find a transitional abstraction like Kubernetes. What is important is that the technologies and topologies between Staging and Production must be identical. I've worked at companies that try to shortchange the layout of the Staging environment to save money, but inevitably this just delays surprises until final Production deployment. For example, load testing and failover tests won't be accurate if the Staging environment is undersized.

last update 2019-05-05