Integrations

CompatibilityKubernetes1.32+OpenShift4.19+Rancher2.10.x+Helm3.2.0+

This page presents the main integration patterns used by TDP Kubernetes components.

It does not replace the component-specific integration pages. Use this page to understand the general concepts, and consult the component page to apply the correct parameters, Secret names, endpoints, values files, and complete examples.

How to Use This Page

Integrations in TDP Kubernetes connect a component to another platform service or to an external service.

These integrations may involve databases, object storage, metadata catalogs, governance services, data catalog tools, messaging systems, external APIs, or other services required by the component.

Not every component uses all of these patterns. The availability of each integration depends on the chart, exposed values, installed version, and environment configuration.

Topic	What it Defines	Where to Detail
External database	Use of a database outside the component chart	Component integration page
Object storage	Connection to S3-compatible storage	Component integration page
Metadata catalog	Use of a catalog for databases, tables, schemas, and data formats	Component integration page
Governance and authorization	Integration with policy and access control services	Component integration or security page
Data catalog and lineage	Registration of metadata, data assets, and traceability	Component integration page
Messaging, events, and connectors	Integration with brokers, topics, connectors, or pipelines	Component integration page

General Principles

Before configuring an integration, validate:

• whether the component chart documents that integration;
• which configuration blocks should be used;
• which Secrets must exist;
• whether the endpoint is internal to the cluster or external;
• whether TLS, authentication, or authorization is required;
• whether the configuration belongs to the component, the integrated service, or both;
• whether jobs, hooks, or automatic initialization steps are associated with the integration.

Internal and External Endpoints

Integrations may use internal Kubernetes addresses or endpoints outside the cluster.

Internal Kubernetes Endpoints

When two components run in the same cluster, communication usually uses Kubernetes Services.

The common format is:

<SERVICE_NAME>.<NAMESPACE>.svc.cluster.local

This address is internal to the cluster and should be used by Pods, jobs, and services that need to communicate with each other.

Generic example:

<POSTGRESQL_SERVICE>.<NAMESPACE>.svc.cluster.local

Replace <SERVICE_NAME> and <NAMESPACE> with the real names from the environment.

External Endpoints

When the integrated service is outside the cluster, use the corresponding external hostname or endpoint.

Examples:

postgresql.empresa.com.br

s3.empresa.com.br

The choice between internal and external endpoint depends on where the service is installed and how it is exposed in the infrastructure.

Credentials and Secrets

Integrations commonly require credentials, such as:

• database username and password;
• object storage access key and secret key;
• connector credentials;
• LDAP bind passwords;
• API tokens;
• certificates, keystores, or truststores.

These credentials should be stored in Kubernetes Secrets whenever the chart allows it.

Generic example:

Terminal input

kubectl -n <NAMESPACE> create secret generic <APPLICATION_NAME>-credentials --from-literal=username='...' --from-literal=password='...'

The way to reference the Secret varies by component. Consult the specific Integration and Security pages before applying the configuration.

Never version passwords

Do not include plain-text passwords in values files committed to Git. Use Secrets or dedicated credential management tools.

External Database

Some components can use an external database instead of an internal, embedded, or chart-provisioned database.

This type of integration commonly requires:

• database hostname;
• port;
• database name;
• username;
• password;
• Secret with credentials;
• parameters for creating or reusing the database;
• connectivity validation between the component and the database.

The exact configuration pattern varies by chart.

Conceptual example:

externalDatabase:
  enabled: true
  host: "<DATABASE_HOST>"
  port: 5432
  database: "<DATABASE_NAME>"
  existingSecret: "<DATABASE_SECRET>"

note

The example above is conceptual only. Use the block documented in the component-specific page.

S3-Compatible Object Storage

Some components can read from or write to S3-compatible storage, such as object storage services inside or outside the cluster.

This type of integration may involve:

• S3 endpoint;
• bucket;
• access key;
• secret key;
• region;
• path-style mode;
• certificates;
• Hadoop/Spark S3A configuration;
• Secret with credentials.

Conceptual example:

s3:
  endpoint: "https://<S3_ENDPOINT>"
  bucket: "<S3_BUCKET>"
  existingSecret: "<S3_SECRET_NAME>"

In some charts, the S3 integration may be a connection managed by the application. In others, it may appear as Hadoop properties, Spark properties, catalog configuration, storage policy, or warehouse configuration.

Consult the component-specific page to confirm the pattern used.

Metadata Catalog

Some components may depend on a metadata catalog to locate databases, tables, schemas, data formats, and physical data locations.

This type of integration usually involves:

• catalog service URI;
• communication protocol;
• Kubernetes namespace and Service;
• warehouse configuration;
• integration with object storage when data is stored in S3 or an equivalent service;
• properties specific to the engine or framework used by the component.

Conceptual internal URI example:

thrift://<SERVICE_NAME>.<NAMESPACE>.svc.cluster.local:<SERVICE_PORT>

The way to declare this integration varies by component.

Governance, Authorization, and Policies

Some components can be integrated with governance or authorization mechanisms.

In these cases, it is important to separate two sides of the configuration:

Side	Meaning
Governance service	Registration of services, policies, users, groups, or permissions
Integrated component	Configuration required to consult or enforce policies at runtime

Registering a service in a governance tool does not, by itself, guarantee that the component is already enforcing policies.

When this type of integration exists, validate:

• whether the service was registered correctly;
• whether policies were created or declared;
• whether users and groups exist;
• whether the integrated component is configured to use the authorization mechanism;
• whether certificates, endpoints, and credentials are correct.

Details should be checked in the component-specific page and, when applicable, in the integration or security page of the governance service.

Data Catalog and Lineage

Some integrations are intended to catalog metadata, register data assets, or track lineage.

This type of integration may involve:

• connectors configured in the catalog tool;
• credentials to access source services;
• read permissions;
• DAGs, jobs, or ingestion pipelines;
• internal or external endpoints of the integrated services.

Configuration may be done by the chart, by the tool interface, by ingestion workflows, or by component-specific automation.

Consult the integration page of the component responsible for the catalog to understand the recommended flow.

Messaging, Events, and Connectors

Messaging and connector integrations usually involve brokers, topics, connectors, credentials, and bootstrap endpoints.

This type of configuration may be used for:

• data ingestion;
• event publishing;
• change data capture;
• system integration;
• streaming pipelines.

Always validate:

• bootstrap endpoint;
• security protocol;
• credentials;
• certificates, when TLS is used;
• read and write permissions;
• connector-specific settings.

Concrete parameters should be applied according to the component integration page.

Integration Flow

Some integrations require configuration in more than one place.

Situation	Attention Point
Service registered in an external tool	The component may still need to be configured to use that service
Secret created in Kubernetes	The chart still needs to reference that Secret correctly
Endpoint provided in configuration	The service must be reachable from the Pod or job that will use the integration
Policy created in a governance service	The component must be prepared to consult or enforce that policy
Catalog configured	The engine or application must use that catalog at runtime

For this reason, always validate the full integration flow, not only the creation of an isolated resource.

Validating an Integration

After configuring an integration, validate it at three levels.

Level	What to Validate
Configuration	Chart values point to the correct endpoints, Secrets, and parameters
Connectivity	The component can reach the integrated service
Functionality	The integration performs the expected operation, such as authenticating, querying, writing, cataloging, or enforcing a policy

Useful commands depend on the component, but commonly include:

Terminal input

kubectl get pods -n <NAMESPACE>

Terminal input

kubectl logs -n <NAMESPACE> <POD_NAME>

Terminal input

kubectl get secret -n <NAMESPACE>

Terminal input

kubectl exec -n <NAMESPACE> <POD_NAME> -- sh -c '...'

Use the component-specific page for complete validation commands.

Troubleshooting

Symptom	Possible Cause
Component cannot connect to the integrated service	Incorrect endpoint, port, namespace, or DNS
Authentication error	Missing Secret, incorrect credential, or user without permission
Certificate error	Missing certificate, incorrect truststore, or hostname not covered by the certificate
Integration created, but with no runtime effect	Only one side of the integration was configured
Integration job fails	Integrated service unavailable, incorrect credentials, or incomplete parameters
Resource does not appear in the catalog tool	Connector was not executed, credential has no permission, or source is unreachable
Configuration appears correct, but has no effect	The component may require complementary configuration in another block or page

Next Steps

To configure a concrete integration, open the corresponding component integration page.

Use this page as a general reference to understand TDP Kubernetes integration patterns. Parameters, examples, and validations should always be confirmed in the component-specific documentation.