Highlights
| Platform | Version | Highlights |
|---|---|---|
| Tecnisys Data Platform | TDP 2.2.0 | - Support for Enterprise Linux 8 and 9 operating systems (Rocky Linux, AlmaLinux, Red Hat, among others). - Apache Ambari with extensive support for Python 3. - Inclusion of HBase REST API. - Improvements in Hive authentication mechanism with LDAP. - Use of OpenJDK8 as the default JDK for Ambari. - Fixed issues with Pydruid library for Kerberos authentication. - Kafka-UI as a graphical interface for managing Kafka clusters. - Fixed jaas_conf of Spark for Kerberized environments. - Added hive.server2.leader.zookeeper.namespace property to Hive. - Added Rolling Upgrade option for new TDP stack versions. - Improvements in the integration of various components with SSL/TLS enabled. - Improvements in the integration of Flink, Iceberg, and Zeppelin components. - Addition of ResourceManager UI in Knox topology. - Flink integration with Avro, JDBC, ORC, Parquet, CSV, Hive, Kafka, and Iceberg. - Spark 3.3.4 integration with Delta 2, Elastic, Iceberg, Kafka, PostgreSQL, Phoenix (HBase), and Solr. - Improvements in Ambari Metrics charts. - Updated components: Delta, Iceberg, Grafana (integrated with Ambari Metrics), Phoenix, and Zeppelin. |
Table A. Platform Highlights
Component Highlights
See also the highlights of the updated components:
| Service | Version | Category | Highlights |
|---|---|---|---|
| Apache Ambari | Ambari 3.0.0.0 | Administration | - Modernization of the interface, making it more intuitive and modern. - Improvements and bug fixes in Ambari Metrics. - Operation with Python 3 on EL 8 and 9 operating systems. |
| Apache Iceberg | Iceberg 1.5.0 | Analytics | - Fixed issue in FileIO where an extra header request was made when reading manifests. - Marked HTTP 502 and 504 status codes as retryable in the REST Client. - Bug fixes and improvements in JDBC Catalog. |
| Apache Phoenix | Phoenix 5.1.3 | Analytics | - Improvements in stability and bug fixes with the JDBC client. - Fixes related to transaction management to ensure greater data consistency and integrity. - Query execution improvements, especially for high data volume scenarios, reducing response time and optimizing resource usage. - Improvements in handling and using secondary indexes to speed up read operations. - Updates to better support integration with Apache Spark, allowing the use of Phoenix as an SQL query layer for HBase data within Spark pipelines. - Greater compatibility with recent versions of HBase. |
| Apache Spark | Spark 3.3.4 | Analytics, Data Science, Graph, Streaming | - Adjustments in the behavior of several SQL functions, such as percentile_disc and percentile_approx(), to correctly handle NULL values. - Improvement in the behavior of SQL functions, including to_number, try_to_number, and handling of columns with GROUPING SETS. - Fixes in the behavior of expression pushdown in DataSource V2 and binary comparison functions. - Improved thread management and structural integrity of CoarseGrainedExecutorBackend. - Fixes in performance issues related to TransportClientFactory and the use of await(). - Adjustments to improve compatibility with Kubernetes and YARN environments, ensuring that resource allocation and token renewal are handled correctly. - Dependency updates, including the ORC version to 1.7.10. |
| Apache Zeppelin | Zeppelin 0.11.0 | Notebook | - Default support for JDK 11, which improves compatibility and performance over previous versions. - Support for the latest versions of Apache Spark (3.5.0) and Apache Flink (1.17). - Python 3.9 is now the default version for the Python interpreter, ensuring better support and performance for Python scripts. - Dynamic creation of forms within notebooks to facilitate user interaction with data. - Improvements in data visualization functionalities, including pivot charts and other interactive tools for data analysis. - Bug fixes and security improvements. |
Table B. Component Highlights