Skip to content

How Is Our Cluster Setup

This section will be a rundown of the way that the mms_cloud_infrastructure repository is configured and the way that all the moving parts interact with each other. It will go over:

  • Directory Structure
  • What is in what directory
  • Conventions and making sense of them
  • How to add a new application
  • How to manage existing applications

Git Ops

Before we continue, it is important to understand methodology that we follow to declare our cluster. When setting up any sort of service, there are two ways that you can go about describing how your service should behave.

  1. Imperative
  2. Declarative

In an imperative setup, you manually set each part of the process, telling the server how configure each part such that you can reach some desired outcome/state. This is equivalent to doing kubectl apply <some_file.yaml> over and over until you get the necessary apps in the desired state.

Our cluster is setup declaratively, meaning we only describe the 'final' state that we want each application to be at. Then trust ArgoCD to figure out how to get there. When looking at any .yaml file in the repository, you should keep in mind that each one is an attempt to describe a state that we want our cluster to be in.

Directory Structure

mms_cloud_infrastructure
├── docs
└── kubernetes
    ├── apps
    └── argocd-management

argocd-management

This directory stores all the manifest and configuration files for ArgoCD itself. The configuration files in here describe how we should interpret the files that we see in the apps directory. It also explains how the CD tool has to be setup when you bootstrap the server.

argocd-management
├── argocd-cm-patch.yaml
├── argocd-cmd-params-cm-patch.yaml
├── argocd-gpg-disable-patch.yaml
├── argocd-manifest.yaml
├── argocd-rbac-cm-patch.yaml
├── argocd-tls-certs-cm-patch.yaml
├── kustomization.yaml
└── unified-automation.yaml

Most of these files do not need to be directly edited by a server maintainer. However, it is good practice to have an understanding of what they do.

apps

This directory contains the configuration of all the applications that we run on the cluster. These are files that ArgoCD reads and then attempts to execute on. 

apps
├── cert-manager
├── container-registry
├── dex
├── example
├── ingress
├── jenkins
├── mms-docs
├── rbac-oidc
├── README.md
├── sealed-secrets
└── traefik

Each application is then further split into something resembling the following:

apps/example
├── config.yaml
├── manifest.yaml
└── values.yaml
ArgoCD is configured to look for these specific files in order to determine what kind of Application you are trying to run 1

The config.yaml is what defines the very basics of the application.

# THIS FILE SHOULD BE NAMED 'config.yaml'
appName: <name>
namespace: <namespace-name>

hasHelm: true
hasManifests: true

helm:
  repoURL: https://<url>
  chart: <chart-name>
  targetRevision: <version-number>        # should be the chart version
You can configure objects to either be built from a Helm Chart, or from a manifest and custom image that you provide. Most of our apps are built using Helm charts. In order to specify the settings that we want to ovveride, we have to author a values.yaml file.

# THIS FILE SHOULD BE NAMED 'values.yaml'
#
# helm charts support overrides. Read through the documentation for
# the helm chart you are using and add any necessary overrides into this file

myCustomOverride: "value of the override"
aVariable: true

If we instead/also want to use a custom manifest to describe our application, we can toggle that with the hasManifests flag. Which will prompt ArgoCD to look for the manifests.yaml file.

# Create a standard kubernetes manifest which can be applied to the cluster
# You will often see commands like 'kubectl apply -f <file>.yaml'. Copy the .yaml file
# into a new project folder and it will be automatically applied by ArgoCD for you

Adding a New Application

Before you add a new application, make sure that you have helm installed so that you can consult the applications helm chart (if it has one) Then you just have to do the following:

  • Make a new folder with the name of your application.

Naming Conventions

It is convention to name each application in lower-kebab-case. This means that the name of the app and the directory it is in, should match that.

  • Copy over the config.yaml, value.yaml and manifest.yaml (if you have a specific manifest you want to apply.)
  • Populate the config.yaml file with the necessary fields that you need.
  • Add a secrets dir if you need to store sealed secrets for your application/namespace.2
  • Add other manifests files as required, making sure to name them appropriately.

Manifest File Conventions

It is convention to have one file per Kubernetes object that you declare, but you can break this if each manifest is small. Exercise your own judgement on when to split up files.

Once you are satisfied with your declaration, you can commit the changes and push them to the repository. ArgoCD will automatically pick up the changes and attempt to apply them on the cluster. To make ArgoCD manually sync your changes, you can click on the Sync button in the Web UI.

Managing Existing Applications

In order to change the behaviour of existing applications, you can simply change the

  1. You can see this configuration in the unified-automation.yaml file. ↩

  2. Consult the structure of some of the other applications like jenkins or dex on how to store secrets ↩