Skip to main content
Version 3.0

Installation via Argo CD

CompatibilityKubernetes1.32+OpenShift4.19+Rancher2.10.x+Helm3.2.0+
tip

Before you start, make sure the prerequisites have been met.

If Argo CD is not yet installed, Step 4 below (Option A, deploy.sh --install) installs it automatically — together with the GitOps bootstrap. If it's already installed, use Option B (deploy.sh -c).

To install the components manually, without GitOps, see the Helm installation.

Overview​

Argo CD installs components from YAML files called Applications. Each TDP component — PostgreSQL, Kafka, Airflow, Trino, etc. — is represented by an Application file that instructs Argo CD to pull the corresponding chart from the Tecnisys registry and install it in the cluster.

These files live in a Git repository. Argo CD continuously monitors that repository and syncs the cluster state with what is declared — without running commands on every change.

Difference between installing with Argo CD and Helm​

CriterionHelm CLIArgo CD (GitOps)
TraceabilityManual commands, no automatic historyEvery change is recorded in Git
AutomationManual execution of each chartContinuous automatic synchronization
RollbackRequires re-running commandsOne click or one command
VisibilityTerminal onlyWeb interface with real-time status
AuditingDifficultComplete, via Git history

GitOps repository structure​

The current implementation uses a directory structure optimized for GitOps:

tdp-gitops/
├── common/                          # Bootstrap resources (AppProject, Secrets, App of Apps)
│   ├── argo-gitops-appproject.yaml
│   ├── argo-gitops-app-of-apps.yaml
│   ├── tdp-devops-repo-secret.yaml
│   └── tdp-registry-secret.yaml
├── available/                       # Application templates (with variables)
│   ├── tdp-postgresql/
│   │   ├── tdp-postgresql.yaml
│   │   └── values.yaml
│   ├── tdp-kafka/
│   └── ...
├── current/                         # Rendered manifests (committed to git)
│   ├── common/
│   ├── tdp-postgresql/
│   └── ...
├── variables.env                    # Environment variables (do not commit)
├── deploy.sh                        # Automation script
└── README.md

How it works:

  • available/ contains templates with variables (${VAR}) — never applied directly.
  • deploy.sh renders available/ + common/ → current/ substituting the variables.
  • current/ is committed to git and monitored by the Argo CD App of Apps.
  • common/ contains the bootstrap resources (AppProject, Secrets, App of Apps).

Essential concepts​

Manifest: A YAML file that describes a Kubernetes or Argo CD resource. In this context, each manifest defines an Argo CD Application — that is, it instructs Argo CD to install a given Helm chart.

Application: The central Argo CD object. It directs Argo CD to pull a given Helm chart from the Registry and install it in a given cluster namespace. Each TDP component is represented by a separate Application.

Repository (in the Argo CD context): The origin of the artifacts Argo CD will use. It can be a Git repository (where the manifests live) or an OCI1 registry (where the Tecnisys Helm Charts live).

Synchronization: The process by which Argo CD compares the state declared in the manifests with the cluster's actual state and applies the required changes.

App of Apps: A pattern that lets you manage all components with a single root Application, which automatically discovers and installs all the others.

Installation order​

When the App of Apps is applied, Argo CD tries to sync all components at the same time. Therefore, components without dependencies must be stable before the dependent ones try to connect to them. The table below shows this order and each chart's dependencies:

OrderChartComponentVersionDependencies
1tdp-postgresqlPostgreSQL17.5.0None
2tdp-kafkaKafka (Strimzi)4.1.0tdp-crds (Strimzi CRDs)
3tdp-hive-metastoreHive Metastore4.0.0tdp-postgresql
4tdp-deltalakeDelta Lake4.0.1None
5tdp-icebergIceberg1.10.0None
6tdp-ozoneApache Ozone2.0.0None
7tdp-rangerRanger2.7.0tdp-postgresql
8tdp-nifiNiFi1.28.0None
9tdp-sparkSpark4.0.2None
10tdp-trinoTrino478None
11tdp-airflowAirflow3.0.2tdp-postgresql
12tdp-jupyterJupyterHub5.3.0tdp-postgresql
13tdp-clickhouseClickHouse25.8.11.66tdp-crds (Altinity CRDs)
14tdp-cloudbeaverCloudBeaver25.3.0tdp-postgresql
15tdp-openmetadataOpenMetadata1.12.4tdp-postgresql
16tdp-supersetSuperset5.0.0tdp-postgresql

The Version column refers to the component version (e.g., Kafka 4.1.0, Airflow 3.0.2). The Helm chart version is 3.0.1 for all TDP charts.

Prerequisites​

ToolRequired for
kubectl (configured for the cluster)deploy, bootstrap
envsubst (gettext package)template rendering
gitGitOps mode (push current/)
helmArgo CD bootstrap (--install)

Step 1 – Get the GitOps repository​

Clone (or fork) the GitOps repository and enter the folder:

Terminal input
git clone <GITOPS_REPO_URL>
cd tdp-gitops
Figure 1 - Cloning the GitOps repository
Figure 1 - Cloning the GitOps repository

Step 2 – Configure environment variables​

Copy the reference file and edit it with your values:

Terminal input
cp variables.env variables.env.local
vim variables.env.local

Required variables in variables.env.local:

variables.env.local
# Namespace where Argo CD is installed
ARGOCD_NAMESPACE=argocd

# Target namespace for the TDP pods
TDP_NAMESPACE=<NAMESPACE>

# Namespace where the Argo CD Applications are created
TDP_PROJECT_NAMESPACE=<NAMESPACE>

# Name of the "App of Apps" Application
TDP_APPLICATIONS=argo-gitops-tdp-project

# URL of this git repository (used by Argo CD to monitor current/)
GIT_REPO_URL=<GITOPS_REPO_URL>
GIT_REPO_USER=<GIT_USERNAME>
GIT_REPO_TOKEN_OR_PASS=<GIT_TOKEN>

# Helm OCI registry with the TDP charts
HELM_CHART_REPO_URL=registry.tecnisys.com.br/tdp/charts
HELM_CHART_VERSION=<TDP_K8S_RELEASE>

# Kubernetes server (use the default for in-cluster)
KUBERNETES_SERVER=https://kubernetes.default.svc

# Helm OCI registry credentials
TECNISYS_HELM_REGISTRY_USER=<REGISTRY_USERNAME>
TECNISYS_HELM_REGISTRY_TOKEN=<REGISTRY_CLI_SECRET>

For HELM_CHART_VERSION, use the current release version: 3.0.1.

warning

variables.env.local is in .gitignore — never commit tokens or passwords.

Step 3 – Configure Argo CD to manage Applications in a separate namespace​

The Applications live in the <NAMESPACE> namespace (set in TDP_PROJECT_NAMESPACE), not in the argocd namespace. Configure Argo CD to manage that namespace through the application.namespaces parameter — see Argo CD configuration → Application namespaces.

Step 4 – Argo CD bootstrap​

The deploy.sh script automates the Argo CD installation and the GitOps bootstrap.

Option A: Full bootstrap — installs Argo CD (recommended)​

Installs Argo CD (tdp-crds + tdp-argo) via Helm OCI, waits for the pods to be ready, renders the templates, and applies the common resources:

Terminal input
./deploy.sh --install -v variables.env.local

What is executed:

  1. helm registry login → authenticates to the OCI registry
  2. helm upgrade --install tdp-crds → installs the Argo CD CRDs
  3. helm upgrade --install tdp-argo → installs Argo CD
  4. kubectl rollout status → waits for argocd-server and application-controller
  5. envsubst on all templates → populates current/
  6. kubectl apply on the files in current/common/:
    • argo-gitops-appproject.yaml — TDP AppProject
    • tdp-devops-repo-secret.yaml — git credential for Argo CD
    • tdp-registry-secret.yaml — Helm OCI registry credential
    • argo-gitops-app-of-apps.yaml — "App of Apps" Application
Figure 2 - Full bootstrap with deploy.sh --install
Figure 2 - Full bootstrap with deploy.sh --install

Option B: GitOps only — Argo CD already installed​

If Argo CD is already installed, just render and apply the common resources:

Terminal input
./deploy.sh -c -v variables.env.local
Figure 3 - Manual bootstrap with deploy.sh -c
Figure 3 - Manual bootstrap with deploy.sh -c

Step 5 – Commit and publish current/ to Git​

The App of Apps reads current/ directly from the git repository. The files must be published:

Terminal input
git add current/
git commit -m "chore: render gitops manifests"
git push origin main

The App of Apps will automatically create all Applications present in current/ with selfHeal: true.

Figure 4 - Publish current/ to the Git repository
Figure 4 - Publish current/ to the Git repository

Step 6 – Follow the synchronization​

Verify the status​

Terminal input
# App-of-Apps Application (argocd namespace)
kubectl get application -n argocd

# Applications created (TDP components namespace)
kubectl get application -n <NAMESPACE>

# Pods in the components namespace
kubectl get pods -n <NAMESPACE>

# Argo CD admin initial password
kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath='{.data.password}' | base64 -d && echo
Figure 5 - Follow the Applications status
Figure 5 - Follow the Applications status

Via the web interface​

Access the Argo CD interface:

Terminal input
kubectl get ingress -n argocd
# or
kubectl get httproute -n argocd
Figure 6 - Finding the exposed host
Figure 6 - Finding the exposed host

Navigate to the Application configured in TDP_APPLICATIONS and follow the progress.

Understanding the Application manifest​

Each component uses a parameterized template. Example (rendered):

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: tdp-postgresql
  namespace: <NAMESPACE>
  labels:
    tecnisys.io/cluster: tdp
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: <NAMESPACE>
  source:
    repoURL: registry.tecnisys.com.br/tdp/charts
    targetRevision: <TDP_K8S_RELEASE>
    chart: tdp-postgresql
    helm:
      valueFiles:
        - values.yaml
  destination:
    server: https://kubernetes.default.svc
    namespace: <NAMESPACE>
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - RespectIgnoreDifferences=true
  ignoreDifferences:
    - group: ""
      kind: Secret
      jqPathExpressions:
        - ".data"

Important fields:

  • metadata.namespace — namespace where the Application is created (TDP_PROJECT_NAMESPACE)
  • spec.project — AppProject that defines the permissions (TDP_NAMESPACE)
  • spec.source.repoURL — OCI registry of the TDP charts
  • spec.source.chart — chart name
  • spec.destination.namespace — namespace where the pods are created (TDP_NAMESPACE)
  • syncPolicy.automated — automatic synchronization
  • ignoreDifferences — ignores changes to Secrets (avoids password flapping)

Component customization​

Each component has a values.yaml in available/<component>/:

Terminal input
# Edit the component values
vim available/tdp-postgresql/values.yaml

# Re-render
./deploy.sh -v variables.env.local -p

# Commit and publish
git add current/tdp-postgresql/
git commit -m "chore: update postgresql values"
git push origin main

deploy.sh: full reference​

Terminal input
./deploy.sh [OPTIONS] [COMPONENTS...]
FlagDescription
--installFirst installation: helm login → tdp-crds → tdp-argo → wait ready → render → apply common
-v FILEUse a custom variables file (default: variables.env)
-pRender available/ → current/ only (without applying)
-cApply only the common resources (current/common/) via kubectl
-aRender + apply components via kubectl (does not apply common)
-hShow help

Examples:

Terminal input
# Full first installation (installs Argo CD + bootstrap)
./deploy.sh --install -v variables.env.local

# Re-render templates and publish (GitOps mode)
./deploy.sh -p -v variables.env.local
git add current/ && git commit -m "chore: render" && git push

# Re-apply only the common resources (AppProject, Secrets, App of Apps)
./deploy.sh -c -v variables.env.local

# Direct deploy of a component (without git push)
./deploy.sh -a tdp-postgresql -v variables.env.local

# Direct deploy of multiple components
./deploy.sh -a tdp-postgresql tdp-trino -v variables.env.local

ArgoCD compatibility​

All TDP charts were adjusted for full Argo CD compatibility:

  • Single-source: Applications use only the OCI source (no multi-source).
  • ignoreDifferences: Secrets ignored to avoid password flapping.
  • Check-or-reuse: credentials reused when they already exist (no rotation on sync).
  • Separate namespace: Applications in TDP_PROJECT_NAMESPACE.
  • Sync waves: correct ordering of dependent resources.

Components with specific Argo CD fixes:

  • tdp-jupyter: password rotation fix in db-create-job
  • tdp-trino: removal of deletable hooks (PVC/Job)
  • tdp-hive-metastore: password rotation fix + security context
  • tdp-superset: Bitnami PostgreSQL randAlphaNum fix
  • tdp-clickhouse: cleanup job disabled by default
  • tdp-airflow: removal of lookup in secrets

Security​

  • variables.env.local is in .gitignore — never commit tokens or passwords.
  • Use Kubernetes Secrets for sensitive data at runtime.
  • Rotate the git and registry tokens regularly.
  • TDP_PROJECT_NAMESPACE must have restricted RBAC.
  • The registry and git Secrets are versioned as templates in common/ (populated from variables.env.local).

Available components​

ComponentDescriptionDependencies
tdp-postgresqlPostgreSQL—
tdp-trinoTrino SQL engine—
tdp-airflowApache AirflowPostgreSQL
tdp-clickhouseClickHouse—
tdp-cloudbeaverCloudBeaver UIPostgreSQL
tdp-deltalakeDelta Lake—
tdp-hive-metastoreHive MetastorePostgreSQL
tdp-icebergApache Iceberg—
tdp-jupyterJupyterHub—
tdp-kafkaApache Kafka (Strimzi)—
tdp-nifiApache NiFi—
tdp-openmetadataOpenMetadataPostgreSQL
tdp-ozoneApache Ozone S3—
tdp-rangerApache Ranger—
tdp-sparkApache Spark—
tdp-supersetApache SupersetPostgreSQL

Footnotes​

  1. OCI (Open Container Initiative) is the modern standard for storing not only container images but also Helm packages. That is why, when registering the registry, you must check the Enable OCI option — without it, Argo CD will try to treat the address as a conventional index.yaml-based Helm repository and the connection will fail. ↩