4 Adding and Deleting Oracle Clusterware Homes

Prerequisite Steps for Adding Oracle Clusterware

The following steps assume that you already have an operative Linux or UNIX environment.

Complete the following steps to prepare the new nodes in the cluster:

1. Make physical connections

   Connect the new nodes' hardware to the network infrastructure of your cluster. This includes establishing electrical connections, configuring network interconnects, configuring shared disk subsystem connections, and so on. See your hardware vendor documentation for details about this step.

2. Install the operating system

   Install a cloned image of the operating system that matches the operating system on the other nodes in your cluster. This includes installing required service patches and drivers. See your hardware vendor documentation for details about this process.

3. Create Oracle users.

   As root user, create the Oracle users and groups using the same user ID and group ID as on the existing nodes.

4. Verify the installation with the Cluster Verification Utility (CVU) using the following steps:

   1. From the /bin directory in the CRS_home on the existing nodes, run the CVU command to verify your installation at the post hardware installation stage as shown in the following example, where node_list is a comma-delimited list of nodes you want in your cluster:

      cluvfy stage -post hwos -n node_list|all [-verbose]
      

      This command causes CVU to verify your hardware and operating system environment at the post-hardware setup stage. After you have configured the hardware and operating systems on the new nodes, you can use this command to verify the node is reachable, for example, to all of the nodes from the local node. You can also use this command to verify user equivalence to all given nodes the local node, node connectivity among all of the given nodes, accessibility to shared storage from all of the given nodes, and so on.

      Note:

      You can only use the all option with the -n argument if you have set the CV_NODELIST variable to represent the list of nodes on which you want to perform the CVU operation.

      See Also:

      Appendix A, "Cluster Verification Utility Reference" for more information

   2. From the /bin directory in the CRS_home on the existing nodes, run the CVU command to obtain a detailed comparison of the properties of the reference node with all of the other nodes that are part of your current cluster environment where ref_node is a node in your existing cluster against which you want CVU to compare, for example, the newly added nodes that you specify with the comma-delimited list in node_list for the -n option, orainventory_group is the name of the Oracle inventory group, and osdba_group is the name of the OSDBA group:

      cluvfy comp peer [ -refnode ref_node ] -n node_list 
      [ -orainv orainventory_group ] [ -osdba osdba_group ] [-verbose]
      

   Note:

   For the reference node, select a node from your existing cluster nodes against which you want CVU to compare, for example, the newly added nodes that you specify with the -n option.

5. Check the installation

   To verify that your installation is configured correctly, perform the following steps:

   1. Ensure that the new nodes can access the private interconnect. This interconnect must be properly configured before you can complete the procedures described in this chapter.

   2. If you are not using a cluster file system, then determine the location on which your cluster software was installed on the existing nodes. Make sure that you have at least 250MB of free space on the same location on each of the new nodes to install Oracle Clusterware. In addition, ensure you have enough free space on each new node to install the Oracle binaries.

   3. Ensure that the Oracle Cluster Registry (OCR) and the voting disk are accessible by the new nodes using the same path as the other nodes use. In addition, the OCR and voting disk devices must have the same permissions as on the existing nodes.

   4. Verify user equivalence to and from an existing node to the new nodes using rsh or ssh on Linux and UNIX systems, or on Window systems make sure that you can run the following command from all of the existing nodes of your cluster where the hostname is the public network name of the new node:

      NET USE \\hostname\C$
      

      You have the required administrative privileges on each node if the operating system responds with:

      Command completed successfully.
      

   After completing the procedures in this section, your new nodes are connected to the cluster and configured with the required software to make them visible to Oracle Clusterware.

   Note:

   Avoid changing host names after you complete the Oracle Clusterware installation, including adding or deleting domain qualifications. Nodes with changed host names must be deleted from the cluster and added back with the new name.

Adding and Deleting Oracle Clusterware Homes on Linux and UNIX Systems

This section explains Oracle Clusterware home addition and deletion on Linux and UNIX systems and it assumes that you have already performed the steps in the "Prerequisite Steps for Adding Oracle Clusterware" section.

For node addition, ensure that you install the required operating system patches and updates on the new nodes. Then configure the new nodes to be part of your cluster at the network level. Use the instructions in this section to extend the Oracle Clusterware home from an existing Oracle Clusterware home to the new nodes

Finally, you can optionally extend the Oracle database software with Oracle RAC components to the new nodes and make the new nodes members of the existing Oracle RAC database. See the node addition procedures described in Oracle Real Application Clusters Administration and Deployment Guide.

This section includes the following topics:

• Adding an Oracle Clusterware Home to a New Node On Linux or UNIX Systems

• Deleting an Oracle Clusterware Home from a Linux or UNIX System

./oifcfg delif –node node2

cat ons.config

./racgons remove_config node2:remote_port

./rootdeletenode.sh node2,node2-number

CRS_home/bin/olsnodes -n

./runInstaller -updateNodeList ORACLE_HOME=CRS_home 
"CLUSTER_NODES={node_to_be_deleted}" 
CRS=TRUE -local

./runInstaller -updateNodeList ORACLE_HOME=CRS_home 
"CLUSTER_NODES={remaining_nodes_list}" 
CRS=TRUE

Adding an Oracle Clusterware Home to a Windows System

This section describes how to add new nodes to Oracle Clusterware using OUI. The OUI requires access to the private interconnect that you checked in the "Prerequisite Steps for Adding Oracle Clusterware" section.

Perform the following steps:

1. On one of the existing nodes go to the CRS_home\oui\bin directory and run the addnode.bat script to start OUI.

2. The OUI runs in the add node mode and the OUI Welcome page appears. Click Next and the Specify Cluster Nodes for Node Addition page appears.

3. The upper table on the Specify Cluster Nodes for Node Addition page shows the existing nodes, the private node names and the virtual IP (VIP) addresses that are associated with Oracle Clusterware. Use the lower table to enter the public, private node names and the virtual hostnames of the new nodes.

4. Click Next and OUI verifies connectivity on the existing nodes and on the new nodes. The verifications that OUI performs include determining whether:

   • The nodes are up

   • The nodes are accessible by way of the network

     Note:

     If any of the existing nodes are down, then you can proceed with the procedure. However, once the nodes are up, you must run the following command on each of those nodes:

     setup.exe -updateNodeList -local 
     "CLUSTER_NODES={available_node_list}"
     ORACLE_HOME=CRS_home
     

     This operation should be run from the CRS_home\oui\bin directory and the available_node_list value is a comma-delimited list of all of nodes currently in the cluster and CRS_home defines the Oracle Clusterware home directory.

   • The virtual hostnames are not already in use on the network

   • The user has write permission to create the Oracle Clusterware home on the new nodes

   • The user has write permission to the OUI inventory in the C:\Program Files\Oracle\Inventory directory

5. If OUI detects that the new nodes do not have an inventory location, then:

   • The OUI automatically updates the inventory location in the Registry key

   If any verifications fail, then OUI redisplays the Specify Cluster Nodes for Node Addition page with a Status column in both tables indicating errors. Correct the errors or deselect the nodes that have errors and proceed. However, you cannot deselect existing nodes; you must correct problems on nodes that are already part of your cluster before you can proceed with node addition. If all of the checks succeed, then OUI displays the Node Addition Summary page.

6. The Node Addition Summary page displays the following information showing the products that are installed in the Oracle Clusterware home that you are extending to the new nodes:

   • The source for the add node process, which in this case is the Oracle Clusterware home

   • The private node names that you entered for the new nodes

   • The new nodes that you entered

   • The required and available space on the new nodes

   • The installed products listing the products that are already installed on the existing the Oracle Clusterware home

   Click Next and OUI displays the Cluster Node Addition Progress page.

7. The Cluster Node Addition Progress page shows the status of the cluster node addition process. The table on this page has two columns showing the phase of the node addition process and the phase's status according to the following:

   This page shows the following three OUI phases:

   • Copy the Oracle Clusterware Home to New Nodes—Copies the Oracle Clusterware home to the new nodes unless the Oracle Clusterware home is on the Oracle Cluster File System.

   • Perform Oracle Home Setup—Updates the Registry entries for the new nodes, creates the services, and creates folder entries.

   • Save Cluster Inventory—Updates the node list associated with the Oracle Clusterware home and its inventory.

   The Cluster Node Addition Progress page's Status column displays In Progress while the phase is in progress, Suspended when the phase is pending execution, and Succeeded after the phase completes. On completion, click Exit to end the OUI session. After OUI displays the End of Node Addition page, click Exit to end the OUI session.

8. From CRS_home\install on node1, run the crssetup.add.bat script.

9. Use the following command to perform an integrated validation of the Oracle Clusterware setup on all of the configured nodes, both the preexisting nodes and the nodes that you have added

   cluvfy comp stage -post crinst -n all [-verbose]
   

The CVU -post crinst stage check verifies the integrity of the Oracle Clusterware components. After you have completed the procedures in this section for adding nodes at the Oracle Clusterware layer, you have successfully extended the Oracle Clusterware home from your existing the Oracle Clusterware home to the new nodes. Proceed to Step 3 to prepare the storage for Oracle RAC on the new nodes.

You can optionally run addnode.bat in silent mode, replacing steps 1 through 6 as follows, where nodeI, nodeI+1, and so on are the new nodes that you are adding:

addnode.bat  "CLUSTER_NEW_NODES={nodeI,nodeI+1,…nodeI+n}"
"CLUSTER_NEW_PRIVATE_NODE_NAMES={node-privI,node-privI+1,…node-privI+n}" 
"CLUSTER_NEW_VIRTUAL_HOSTNAMES={node-vipI,node-vipI+1,…,node-vipI+n}"

You can alternatively specify the variable=value entries in a response file and run addnode as follows:

addnode.bat  -responseFile filename

Command-line values always override response file values.

Run crssetup.add.bat on the local node, or on the node on which you were performing this procedure.

After you have completed the procedures in this section for adding nodes at the Oracle Clusterware layer, you have successfully extended the Oracle Clusterware home from your existing the Oracle Clusterware home to additional nodes.

Deleting an Oracle Clusterware Home from a Windows System

This section describes how to delete Oracle Clusterware from Windows systems. These procedures assume that an Oracle Clusterware home is installed on node1 and node2, and that you want to delete node2 from the cluster.

This section contains the following topics:

• Deleting an Oracle Clusterware Home Using OUI in Interactive Mode

• Deleting an Oracle Clusterware Home Using OUI in Silent Mode