Security — Delta Lake
Chart
Compatibility
Feature Support
The tdp-deltalake chart runs maintenance jobs (VACUUM, OPTIMIZE) against S3/MinIO storage. Bucket access credentials are provided via the Kubernetes Secret s3-credentials, which the CronJobs read using env.secretKeyRef.
Credentials and Secrets
Create the Secret in the namespace before deploying:
kubectl -n <NAMESPACE> create secret generic s3-credentials \
--from-literal=access-key="<ACCESS_KEY>" \
--from-literal=secret-key="<SECRET_KEY>"
| Key | Description |
|---|---|
access-key | Access key for the S3/MinIO bucket |
secret-key | Secret key for S3 authentication |
The Secret name must be exactly s3-credentials. The chart CronJobs reference this name directly.
S3 endpoint configuration
The S3/MinIO endpoint is configured in the maintenance.spark.config block:
maintenance:
spark:
config:
"spark.hadoop.fs.s3a.endpoint": "http://<S3_ENDPOINT>:<S3_PORT>"
"spark.hadoop.fs.s3a.path.style.access": "true"
Replace <S3_ENDPOINT> with the internal S3 service address in the cluster. For the Ozone instance deployed with TDP, the default address follows the pattern:
http://<RELEASE_NAME>-s3g-rest.<NAMESPACE>.svc.cluster.local:9878
Best practices
| Aspect | Recommendation |
|---|---|
| Credentials | Do not store access-key and secret-key in a Git repository |
| Rotation | Recreate the Secret and wait for the next CronJob execution cycle |
| Least privilege | Configure credentials with access only to the buckets used by the maintenance jobs |
Troubleshooting
| Problem | Likely cause | Fix |
|---|---|---|
| CronJob fails with authentication error | Secret missing or keys with incorrect name | Check Secret s3-credentials in the namespace |
| S3 endpoint unreachable | Incorrect URL or port | Test connectivity with kubectl exec inside the namespace |
| Permission denied on bucket | Credentials without access to the configured path | Review permissions in MinIO/Ozone |
For the complete list of parameters, use helm show values for the installed chart version.