Engineering principles

I offer the following engineering principles to start with:

• Zero-touch automation for everything (if it’s manual – and you have to do it multiple times a month – it should be automated)
• Project-agnostic solutions (defined in the configuration to avoid re-development for new projects, any tool/module should be reusable)
• IaC (infrastructure should be immutable where possible and defined as code; provisioning tools should be reusable)
• Continuous delivery (CD) with continuous integration (CI) (common approaches and environments across your delivery cycle; any service should be deployable immediately)
• Reliability and security validated at every release (penetration testing, chaos testing, and more should be added to the CI/CD pipeline; always identify the point of flavors at your earliest)
• Be data-driven (real-time data should be utilized to make decisions)

To fully realize your engineering goals and adhere to your principles without compromise, you should make “immutable IaC” a priority objective.

To enable this, I would recommend the following IaC principles:

• Systems can be easily reproduced
• Systems are immutable
• Systems are disposable
• Systems are consistent
• Processes are repeatable
• Code/config are version-controlled

Once you have defined your goals, it’s time for you to choose the right tools for the job. To do that, you must ensure these tools are allowed to utilize the following:

• Declarative orchestration framework(s):
  • The declarative orchestration approach uses structural models that describe the desired application structure and state. These are interpreted by a deployment engine to enforce this state.
  • It enables us to define the end state and interact in a declarative manner, thus making managing the application less resource-intensive (faster speed to market and cheaper costs).

  The following is an example Terraform file (main.tf):

  provider "aws" {
    region = "us-west-2"
  }
  # Create an S3 bucket
  resource "aws_s3_bucket" "my_bucket" {
    bucket = "my-unique-bucket-name"
    acl    = "private"
  }
  # Create an EC2 instance
  resource "aws_instance" "my_instance" {
    ami           = "ami-0c55b159cbfafe1f0" # This is an example Amazon Linux 2 AMI ID; use the appropriate AMI ID for your needs
    instance_type = "t2.micro"
    tags = {
      Name = "MyInstance"
    }
  }

• Declarative resource definition:
  • In a declarative style, you simply say what resources you would like, and any properties they should have so that you can create and deploy an entire infrastructure declaratively. For example, you can deploy not only agents (or sidecars) but also the network infrastructure, storage systems, and any other resources you may need.
  • This enables us to define what our infrastructure resources should look like and force the orchestrator to create it (focus on the how while leveraging declarative orchestration).

  The following is an example that uses Kubernetes, which is a popular container orchestration platform that exemplifies the concept of declarative resource definition. In this example, we’ll define a Deployment for a simple web server and a Service to expose it.

  Here’s a YAML file (deployment-and-service.yaml) for Kubernetes:

  # Deployment definition to create a web server pod
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: my-web-server
  spec:
    replicas: 2
    selector:
      matchLabels:
        app: web-server
    template:
      metadata:
        labels:
          app: web-server
      spec:
        containers:
        - name: nginx
          image: nginx:1.17
          ports:
          - containerPort: 80
  ---
  # Service definition to expose the web server
  apiVersion: v1
  kind: Service
  metadata:
    name: my-web-service
  spec:
    selector:
      app: web-server
    ports:
      - protocol: TCP
        port: 80

• Idempotency:
  • This allows you to create and deploy an entire infrastructure declaratively. For example, you can deploy not only agents (or sidecars) but also the network infrastructure, storage systems, and any other resources you may need. Idempotency is the property that an operation may be applied multiple times with the result not differing from the first application. Restated, this means multiple identical requests should have the same effect as a single request.
  • Idempotency enables the same request to be sent multiple times but the result given is always the same (same as declared, never different).
• No secrets and environment config in code:
  • The main cloud providers all have a secure way to manage secrets. These solutions provide a good way to store secrets or environment config values for the application you host on their services.
  • Everything should be self-served and manageable in a standardized manner and therefore secrets and configs must be declarative and well defined to work with the aforementioned requirements.
• Convention over configuration:
  • Also known as environment tag-based convention over configuration, convention over configuration is a simple concept that is primarily used in programming. It means that the environment in which you work (systems, libraries, languages, and so on) assumes many logical situations by default, so if you adapt to them rather than creating your own rules each time, programming becomes an easier and more productive task.
  • This means that developers have to make fewer decisions when they’re developing and there are always logical default options. These logical default options have been created out of convention, not configuration.
• Automation scripts packaged into an image:
  • This enables immutability and encourages sharing. No longer is a script located on a server and then has to be copied to others – instead, it can be shipped just like the rest of our code, enabling scripts to be available in a registry rather than dependent on others.

Thanks to the amazing progress in this field in the past 10+ years, customer expectations are sky-high when it comes to modern solutions. As we established earlier, if content does not load in under two seconds, it is considered to be slow. If you have to wait longer than 3 to 5 seconds, you are likely to abandon it. This is very similar to availability and customer happiness. When we talk about customer happiness (which evolved from customer experience), a concept you cannot measure and therefore cannot be data-driven, setting the right goals/objectives can be crucial to how you design your solutions.