Integrations — ClickHouse

ChartVersion3.0.1TypeapplicationAppVersion25.8.11.66

CompatibilityKubernetes1.32+OpenShift4.19+Rancher2.10.x+

This page focuses on tdp-clickhouse integrations with S3/MinIO and other tools in the TDP ecosystem.

Aspects of passwords, LDAP and access control are in Security — ClickHouse.

Integrations overview

S3 / MinIO

Chart supports two different levels of integration with S3:

1. Chart integration with the bucket via TDP-Settings.s3Connection
2. ClickHouse's own storage policy via tdp-clickhouse.clickhouse.extraConfig

These two levels are complementary: the first helps prepare the bucket; the second teaches ClickHouse to use S3 as a storage disk.

Bucket via TDP-Settings.s3Connection

When s3Connection.enabled: true, the chart executes a Job that:

• tries to create the bucket, if it does not already exist;
• uses the configured credentials while running the bucket preparation Job;
• creates a Secret (tdp-clickhouse-s3-connection) with the connection details.

TDP-Settings:
  s3Connection:
    enabled: true
    bucket: "<S3_BUCKET>"
    accessKey: "<S3_ACCESS_KEY>"
    secretKey: "<S3_SECRET_KEY>"
    uri: "https://<S3_ENDPOINT>"
    selfSigned: false

Credentials

Store accessKey and secretKey in a separate values file, outside of Git, or in an existing Kubernetes Secret. Never include them directly in the repository.

S3 storage policy in ClickHouse

If you want ClickHouse to store data in S3/MinIO, add the XML configuration in tdp-clickhouse.clickhouse.extraConfig.

tdp-clickhouse:
  clickhouse:
    extraConfig: |
      <yandex>
        <storage_configuration>
          <disks>
            <default>
              <keep_free_space_bytes>1073741824</keep_free_space_bytes>
            </default>
            <s3>
              <type>s3</type>
              <endpoint>https://<S3_ENDPOINT>/<S3_BUCKET>/</endpoint>
              <access_key_id><S3_ACCESS_KEY></access_key_id>
              <secret_access_key><S3_SECRET_KEY></secret_access_key>
              <use_https>1</use_https>
              <skip_hostname_verification_for_https>1</skip_hostname_verification_for_https>
              <bucket><S3_BUCKET></bucket>
              <path>data/</path>
              <use_environment_credentials>false</use_environment_credentials>
              <path_style>true</path_style>
            </s3>
          </disks>
          <policies>
            <default>
              <volumes>
                <main>
                  <disk>default</disk>
                </main>
              </volumes>
            </default>
            <s3>
              <volumes>
                <main>
                  <disk>s3</disk>
                </main>
              </volumes>
            </s3>
          </policies>
        </storage_configuration>
        <openSSL>
          <client>
            <loadDefaultCAFile>false</loadDefaultCAFile>
            <verificationMode>none</verificationMode>
            <invalidCertificateHandler>
              <name>AcceptCertificateHandler</name>
            </invalidCertificateHandler>
          </client>
        </openSSL>
      </yandex>

After applying this policy, you can create tables using storage_policy = 's3':

CREATE TABLE my_table (id UInt64, value String)
ENGINE = MergeTree()
ORDER BY id
SETTINGS storage_policy = 's3';

TLS and certificates

In the example above, verificationMode: none and AcceptCertificateHandler should be treated as an example for controlled environments. In production, prefer proper TLS validation whenever the S3 endpoint supports trusted certificates.

Main S3 Parameters

Parameter	Description	Standard
TDP-Settings.s3Connection.enabled	Enable S3 Integration	false
TDP-Settings.s3Connection.bucket	Bucket name	—
TDP-Settings.s3Connection.accessKey	Access key	—
TDP-Settings.s3Connection.secretKey	Secret Key	—
TDP-Settings.s3Connection.uri	Endpoint S3	—
TDP-Settings.s3Connection.selfSigned	Accept self-signed certificate	false

Consistency between Job S3 and extraConfig

If you use both mechanisms at the same time, keep the following values consistent:

• bucket
• endpoint
• accesskey
• secret key
• TLS/certificate policy

This avoids situations where the bucket is created with one endpoint, but ClickHouse tries to write to a different endpoint.

Other ecosystem integrations

CloudBeaver

CloudBeaver can point to ClickHouse as a datasource. The configuration is done in the tdp-cloudbeaver chart — see Integrations — CloudBeaver.

Superset

Superset can query ClickHouse via the SQLAlchemy driver. The configuration is done in the tdp-superset chart — see Integrations — Superset.

Trino

Trino can consume data from ClickHouse via its own connector. The configuration is done in the tdp-trino chart — see Integrations — Trino.

Combining value files

Terminal input

helm upgrade --install <RELEASE_NAME> \
  oci://registry.tecnisys.com.br/tdp/charts/tdp-clickhouse \
  -n <NAMESPACE> \
  -f my-values.yaml \
  -f values-integration.yaml