Skip to main content
Version: Next

Update Prerequisites

Below we present the minimum prerequisites for updating a TDP Cluster, which may vary according to the desired services or the technical and organizational needs of each environment.

If needed, request support at Support Area.

Disk Space​

The update requires the installation of new packages, which demand additional disk space, either in the package repository or in the directories of the Cluster machines designated for binaries, configuration files, and libraries. For a safe update without incidents, we recommend a minimum of 100 GB of free disk space.

warning

Before installing the packages of the new version, check the available disk space to avoid file corruption, operation restart, among other problems.

Communication Network​

Communication between the Cluster machines and the package repository must be guaranteed, whether through a local network or the internet. The speed and stability of the network directly influence the execution time and integrity of the operation.

Auto-Start Function Disabled​

Before starting the component update, it is important to disable the Auto-Start function of services in Ambari. For this, follow these steps:

  • Select the Service Auto-Start option from the Ambari sidebar menu.

  • Change the switch to Disabled in the Auto Start Settings option.

    Figure 5 - Disabling Auto Start
    Figure 5 - Disabling Auto Start

  • Click Save to save the change and confirm the operation.

    note

    Once the update is completed, the Auto Start function can be enabled again.

Workloads Finished​

We recommend that the main workloads be completed before the update. This avoids interrupting ongoing processes, ensuring data integrity and continuity of operations.

High Availability Services​

The Rolling Upgrade type update is a process that allows updating a service without interrupting the operation of the Cluster. For this, it is necessary that the services are configured for high availability, ensuring the continuity of operations.

Backup of Metadata Databases​

Various components of the TDP platform use metadata databases to store information about configurations, operations, among others.

Before starting the update, it is recommended to perform a backup of the metadata databases, thus ensuring the possibility of restoration in case of failures.

Additional Settings in Apache Ambari for Large Clusters​

In a large Cluster, with dozens or hundreds of machines, some additional settings in Apache Ambari may be necessary to ensure the success of the update.

Adjusting the Timeout for Package Installation​

In a large Cluster, installing packages through Ambari can take a considerable amount of time. Therefore, to avoid timeout issues, increase the value of the agent.package.install.task.timeout parameter, located in the configuration file /etc/ambari-server/conf/ambari.properties of the Ambari Server machine. For this, follow these steps:

  1. Open the Ambari Server configuration file with a text editor:

    Terminal input
        vim /etc/ambari-server/conf/ambari.properties

    1.1. Adjust the value of the agent.package.install.task.timeout property. For example, to 1 hour (3600 seconds):

    Terminal input
         agent.package.install.task.timeout=3600

    1.2. Save the change and close the configuration file of the Ambari Server.

    Figura 6 - Adjust timeout
    Figura 6 - Adjust timeout

  2. Restart the Ambari Server service:

    Terminal input
    ambari-server restart
    Figura 6 - Restart Ambari
    Figura 6 - Restart Ambari

Adjusting the NameNode Restart Time​

In a large Cluster, the process of starting the NameNode can take a significant amount of time. The startup time depends not only on the available computing resources but also on the volume of data and network parameters.

To ensure that Ambari's requests to start the NameNode do not exceed the timeout during an update, configure the NameNode restart timeout parameter in Ambari, upgrade.parameter.nn-restart.timeout, in the file /etc/ambari-server/conf/ambari.properties of the Ambari Server machine.

note

If the parameter upgrade.parameter.nn-restart.timeout does not exist in the configuration file, add it.

Initially, add 10% to the time (in seconds) normally required to restart the NameNode. Although there is no standard method to determine an appropriate value, the following guidance can be used. For example, 660 seconds (11 minutes) if the normal restart time is 600 seconds (10 minutes). For this, follow these steps:

  1. Open the Ambari Server configuration file with a text editor:

    Terminal input
    vim /etc/ambari-server/conf/ambari.properties

    1.1. Adjust the value of the property upgrade.parameter.nn-restart.timeout:

    Terminal input
    upgrade.parameter.nn-restart.timeout=660

    1.2. Save the change and close the configuration file of the Ambari Server.

  2. Restart the Ambari Server service:

    Terminal input
    ambari-server restart
    Figura 6 - Adjust namenode
    Figura 6 - Adjust namenode