Skip to main content
Version Next

Integrations — Apache Ozone

Integration overview​

In the tdp-ozone chart, the S3 Gateway REST is accessible within the cluster at:

http://<release>-s3g-rest.<namespace>.svc.cluster.local:9878

For Ingress, use hostnames that your DNS can resolve (in lab environments you may see examples such as ozone-s3.local).

Spark (s3a)​

spark.hadoop.fs.s3a.endpoint=http://<release>-s3g-rest.<namespace>.svc.cluster.local:9878
spark.hadoop.fs.s3a.access.key=<AWS_ACCESS_KEY_ID>
spark.hadoop.fs.s3a.secret.key=<AWS_SECRET_ACCESS_KEY>
spark.hadoop.fs.s3a.path.style.access=true
spark.hadoop.fs.s3a.connection.ssl.enabled=false

Trino (native S3)​

fs.native-s3.enabled=true
s3.endpoint=http://<release>-s3g-rest.<namespace>.svc.cluster.local:9878
s3.region=us-east-1
s3.aws-access-key=<AWS_ACCESS_KEY_ID>
s3.aws-secret-key=<AWS_SECRET_ACCESS_KEY>
s3.path-style-access=true

Example in a Trino catalog (illustrative values)​

tdp-trino:
  catalogs:
    iceberg: |
      connector.name=iceberg
      iceberg.catalog.type=hive_metastore
      hive.metastore.uri=thrift://<hive-metastore-host>.<namespace>.svc.cluster.local:9083
      fs.native-s3.enabled=true
      s3.endpoint=http://<release>-s3g-rest.<namespace>.svc.cluster.local:9878
      s3.region=us-east-1
      s3.aws-access-key=<AWS_ACCESS_KEY_ID>
      s3.aws-secret-key=<AWS_SECRET_ACCESS_KEY>
      s3.path-style-access=true

Adjust the catalog name and Hive Metastore service name to match your actual chart/release.

Test with AWS CLI​

Terminal input
aws s3 mb s3://test-bucket --endpoint-url=http://<endpoint>
aws s3 ls --endpoint-url=http://<endpoint>

Use <endpoint> as the S3 Gateway URL (internal or via Ingress), for example http://<release>-s3g-rest.<namespace>.svc.cluster.local:9878.