| Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Administration and Deployment Guide 10g Release 2 (10.2) Part Number B14197-02 |
|
|
View PDF |
| Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Administration and Deployment Guide 10g Release 2 (10.2) Part Number B14197-02 |
|
|
View PDF |
This chapter describes how to add and delete nodes and instances in Oracle Real Application Clusters (RAC) databases on Windows-based systems. The preferred method to add instances to RAC databases is to use the Oracle cloning procedures that are described in the Oracle Universal Installer and OPatch User's Guide. Cloning enables you to copy images of the Oracle Clusterware and RAC software onto the other nodes that have identical hardware and software. If you do not want to use the cloning process, then you can use the manual procedures that are described in this chapter to add nodes and instances. The topics in this chapter are:
Cloning Oracle Clusterware and RAC Software in Grid Environments
Quick-Start Node and Database Instance Addition and Deletion Procedures
Detailed Node and Database Instance Addition and Deletion Procedures
|
Note: For all of the add node and delete node procedures, temporary directories such as%TEMP% or C:\Temp should not be shared directories. If your temporary directories are shared, then set your temporary environment variable, such as %TEMP%, to a non-shared location on a local node. In addition, use a directory that exists on all of the nodes. |
|
Note: The entries forCRS_home and Oracle_home as used in this chapter refer to substitutes for either environment variables or full path names for the Oracle Clusterware and Oracle home with RAC respectively. |
The preferred method for extending your RAC environment is to use the Oracle cloning procedures in the Oracle Universal Installer and OPatch User's Guide. Only use the procedures in this chapter if you choose not to use Oracle cloning.
|
See Also: "Cloning Oracle Clusterware and RAC Software in Grid Environments" for a summary about Oracle cloning |
This section explains node and instance addition and deletion using procedures that are presented in a quick-start format. For node addition, these procedures require that you install the required operating system patches and updates on the new nodes. Then configure the new nodes to be part of your cluster at the network level. Extend the Oracle Clusterware home from an existing Oracle Clusterware home to the new nodes and then extend the Oracle database software with RAC components to the new nodes. Finally, make the new nodes members of the existing RAC database. This section includes the following topics:
|
Note: For all of the procedures in this chapter, it is very important that you perform each step in the order shown. |
|
See Also: Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide for Microsoft Windows for procedures about using the Database Configuration Assistant (DBCA) to create and delete RAC databases |
Use one of the procedures described in this section to use the OUI to add an Oracle Clusterware home to a node that is going to be part of your RAC cluster. These procedures assume that you are adding node2 and that you have already successfully installed Oracle Clusterware on node1 in a non-shared home, where CRS_home represents the successfully installed home.
Adding an Oracle Clusterware Home to a New Node Using the OUI in Interactive Mode
Adding an Oracle Clusterware Home to a New Node Using the OUI in Silent Mode
|
Note: Oracle recommends that you back up your voting disk and Oracle Cluster Registry files after you complete any node addition or deletion procedures. |
Use the following procedure to add an Oracle Clusterware home to a new node, node2, to your cluster with an interactive OUI session.
Go to CRS_home\oui\bin on node1 and run the addnode.bat script.
The OUI displays the Node Selection page on which you must specify the node, private node name, and the VIP address for the new node that you are adding and click Next.
Verify the contents that the OUI displays on the summary page and click Next. This installs the Oracle Clusterware home with RAC on the new node.
From CRS_home\install on node1, run the crssetup.add.bat script.You
Use the following procedure to add an Oracle Clusterware home to a new node, node2, to your cluster with a silent OUI session.
Go to CRS_home\oui\bin on node1 and run the addnode.bat script as follows:
addnode.bat –silent "CLUSTER_NEW_NODES={node2}"
"CLUSTER_NEW_PRIVATE_NODE_NAMES={node2-priv}"
"CLUSTER_NEW_VIRTUAL_HOSTNAMES={node2-vip}"
Perform step 4 from the previous section about using the OUI interactively under the heading "Adding an Oracle Clusterware Home to a New Node Using the OUI in Interactive Mode" to add node2 to the cluster.
Use one of the following procedures to add an Oracle home to a new node using the OUI. These procedures assume that you want to extend an existing Oracle home with RAC on node1 to node2. Node2 should already be a member node of the cluster to which node1 belongs. The contents of these procedures assume that you have successfully installed RAC on node1 in a non-shared home and that Oracle_home represents the successfully installed Oracle home. This section describes the following procedures:
Adding an Oracle Home with RAC to a New Node Using the OUI in Interactive Mode
Adding an Oracle Home with RAC to a New Node Using the OUI in Silent Mode
|
Note: Oracle recommends that you back up your voting disk and Oracle Cluster Registry files after you complete any node addition or deletion procedures. |
Use the following procedure to add an Oracle home to a new node, node2, using the OUI in interactive mode.
Go to Oracle_home\oui\bin and run the addnode.bat script.
The OUI displays the Node Selection page on which you should select the nodes that you want to add and then click Next.
Verify the contents that the OUI displays on the Summary page and click Next. This installs the Oracle home with RAC on the new node.
On node2, run NETCA to add a Listener.
To add a database instance to the new node, use Enterprise Manager or DBCA as described under the heading "Step 5: Adding Database Instances to New Nodes"
Use the following procedure to add an Oracle home to a new node, node2, using the OUI in interactive mode.
Go to Oracle_home\oui\bin and run the addnode.bat script as follows:
addnode.bat –silent "CLUSTER_NEW_NODES={node2}"
Perform steps 4 and 5 from the previous section for using the OUI in interactive mode under the heading "Adding an Oracle Home with RAC to a New Node Using the OUI in Interactive Mode" to add the Listener and to create the database instance for the new node.
|
Note: The same steps are applicable for node additions using a shared home. |
Use the following procedures to delete nodes at the database Oracle home layer on Windows-based systems. These procedures assume that a RAC database home is installed on both node1 and node2 and that you want to delete the database home on node2. Also assume that Oracle_home represents the location of this home on the nodes. The procedures in this section are:
Deleting an Oracle Home with RAC Using the OUI in Interactive Mode
Deleting an Oracle Home with RAC Using the OUI in Silent Mode
|
Note: Oracle recommends that you back up your voting disk and Oracle Cluster Registry files after you complete any node addition or deletion procedures. |
Use the following procedures to use the OUI to delete an Oracle home:
Remove any instances that are on node2 with Enterprise Manager or DBCA as described under the heading "Step 1: Deleting Instances from Real Application Clusters Databases". Only delete the database instance on node2 that is associated with the Oracle home that you are deleting. Other instances may exist on node2 that you do not want to delete.
If you use ASM and this is the home from which the ASM instance runs, then perform the ASM instance clean-up procedures under the heading "Step 3: ASM Instance Clean-Up Procedures for Node Deletion".
If this is the home from which the node-specific listener runs, then use NETCA to remove the listener and its CRS resources. If necessary, re-create this listener in other homes.
On the node to be deleted, which in this example is node2, run the following command from Oracle_home\oui\bin:
setup.exe –updateNodeList ORACLE_HOME=Oracle_home
"CLUSTER_NODES={node2}" –local
On the node to be deleted, node2 in this case:
If the home is a non-shared home, then de-install the database by running the setup.exe script from Oracle_home\oui\bin.
If the home is a shared home then do not perform a de-install. Instead, perform the following steps on the node to be deleted:
Perform a detach home on node2 by running the following command from Oracle_home\oui\bin on node2:
setup.exe –detachHome –silent ORACLE_HOME=Oracle_home
On node 2, stop and delete any services that are associated with this Oracle home. In addition, delete any Registry entries and path entries that are associated with this Oracle home. Also, delete all of the start menu items associated with this Oracle home.
On node1, or in the case of an installation with more than two nodes on any node other than the node to be deleted, run the following command from Oracle_home\oui\bin where node_list is a comma-delimited list of nodes that are to remain part of the RAC installation:
setup.exe –updateNodeList ORACLE_HOME=Oracle_home "CLUSTER_NODES={node_list}"
Use the following procedures to delete an Oracle home by using the OUI in silent mode:
Perform steps 1 through 4 from the previous OUI procedure for node deletion.
On node2, run the OUI by running the setup.exe script from Oracle_home\oui\bin:
If you have a non-shared home, then de-install the Oracle home as follows:
setup.exe –silent -deinstall "REMOVE_HOMES={Oracle_home}"
If you have a shared home, then do not perform a de-install. Instead, perform the following steps on the node that you want to delete:
Run the following command from Oracle_home\oui\bin to detach the Oracle home on node2:
setup.exe –detachHome –silent ORACLE_HOME=Oracle_home
On node 2, stop and delete any services that are associated with this Oracle home. In addition, delete any Registry entries and path entries that are associated with this Oracle home. Also, delete all of the start menu items associated with this Oracle home.
Perform step 6 from the previous OUI procedure to update the node list.
Use the following procedures to delete nodes at the Oracle Clusterware layer on Windows-based systems. These procedures assume that an Oracle Clusterware home is installed on node1 and node2, and that you want to delete node2 from the cluster. The topics in this section are:
Deleting an Oracle Clusterware Home Using the OUI in Interactive Mode
Deleting an Oracle Clusterware Home Using the OUI in Silent Mode
|
Note: Oracle recommends that you back up your voting disk and Oracle Cluster Registry files after you complete any node addition or deletion procedures. |
Perform the following procedure to use the OUI to delete a node at the clusterware layer:
Perform the delete node operation for database homes as described in the section titled "Deleting an Oracle Home with RAC Using the OUI in Interactive Mode".
If you did not run the oifcfg command with the -global option, then from node1 at Well, run the following command:
oifcfg delif –node node2
From node1 in the CRS_home\bin directory run the following command where remote_port is the ONS remote port number:
racgons remove_config node2:remote_port
You can determine the remote port by viewing the contents of the file, CRS_home\opmn\conf\ons.config.
Run srvctl to stop and remove the nodeapps from node 2. From CRS_home/bin run the following commands:
srvctl stop nodeapps –n node2 srvctl remove nodeapps –n node2
On node1, or on any node that is not being deleted, run the following command from CRS_home\bin where node_name is the node to be deleted and node_number is the node's number as obtained from the output from the olsnodes -n command:
crssetup del –nn node_name,node_number
On the node(s) to be deleted (node2 in this case), run the following command from CRS_home\oui\bin:
setup.exe –updateNodeList ORACLE_HOME=CRS_home
"CLUSTER_NODES={node2}" CRS=TRUE –local
On node2, using OUI setup.exe from CRS_home\oui\bin:
If you have a non-shared home, then de-install the Oracle Clusterware installation by running the setup.exe script from CRS_home\oui\bin.
If you have a shared home, then do not perform a de-installation. Instead, perform the following steps on the node that you want to delete:
Run the following command from CRS_home\oui\bin to detach the Oracle Clusterware home on node2:
setup.exe –detachHome –silent ORACLE_HOME=CRS_home
On node2, stop and delete any services that are associated with this Oracle Clusterware home. In addition, delete any Registry entries and path entries that are associated with this Oracle Clusterware home. Also, delete all of the start menu items associated with this Oracle Clusterware home. Delete the central inventory as well as the non-Oracle home files under C:\WINDOWS\system32\drivers.
On node1, or in the case of a multiple node installation, on any node other than the one to be deleted, run the following command from CRS_home\oui\bin where node_list is a comma-delimited list of nodes that are to remain part of the Oracle clusterware:
setup.exe –updateNodeList ORACLE_HOME=CRS_home "CLUSTER_NODES={node_list}" CRS=TRUE
Use the following procedure to delete an Oracle Clusterware home by using the OUI in silent mode:
Perform steps 1 through 6 from the previous section for using the OUI interactively under the heading "Deleting an Oracle Clusterware Home Using the OUI in Interactive Mode" to delete nodes at the Oracle Clusterware layer.
On node2, using OUI setup.exe from CRS_home\oui\bin:
If you have a non-shared home, then de-install the Oracle Clusterware home as follows:
setup.exe –silent -deinstall "REMOVE_HOMES={CRS_home}"
If you have a shared home, then do not perform a de-installation. Instead, perform the following steps on the node that you want to delete:
Run the following command from CRS_home\oui\bin to detach the Oracle Clusterware home on node2:
setup.exe –detachHome –silent ORACLE_HOME=CRS_home
On node 2, stop and delete any services that are associated with this Oracle Clusterware home. In addition, delete any Registry entries and path entries that are associated with this Oracle Clusterware home. Also, delete all of the start menu items associated with this Oracle Clusterware home. Delete the central inventory as well as the non-Oracle Clusterware home files under C:\WINDOWS\system32\drivers.
Perform step 8 from the previous OUI procedure to update the node list.
This section provides detailed procedures for adding and deleting nodes to clusters. The details of these steps appear in the following sections:
Step 2: Extending Oracle Software to New Nodes at the Oracle Clusterware Layer
Adding Nodes that Already Have Clusterware and Oracle Software to a Cluster
Step 1: Deleting Instances from Real Application Clusters Databases
Step 2: Deleting Nodes from Real Application Clusters Databases
|
See Also: Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide for Microsoft Windows for procedures about using the Database Configuration Assistant (DBCA) to create and delete RAC databases |
This section explains how to add nodes to clusters using detailed manual procedures. Do this by setting up the new nodes to be part of your cluster at the network level. Then extend the Oracle Clusterware home from an existing Oracle Clusterware home to the new nodes and then extend the Oracle database software with RAC components to the new nodes. Finally, make the new nodes members of the existing RAC database.
If the nodes that you are adding to your cluster do not have Oracle Clusterware or Oracle software installed on them, then complete the following five steps. The procedures in these steps assume that you already have an operative Windows-based RAC environment. The details of these steps appear in the following sections.
To add a node to your cluster when the node is already configured with Oracle Clusterware and Oracle software, follow the procedure described in "Adding Nodes that Already Have Clusterware and Oracle Software to a Cluster".
|
See Also: Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide for Microsoft Windows for procedures about using the DBCA to create and delete RAC databases |
Complete the following procedures to connect the new nodes to the cluster and to prepare them to support your RAC database:
Connect the new nodes' hardware to the network infrastructure of your cluster. This includes establishing electrical connections, configuring network interconnects, configuring shared disk subsystem connections, and so on. Refer to your hardware vendor documentation for details about this step.
As the Windows Administrator, install a cloned image of the operating system that matches the operating system on the other nodes in your cluster. This includes installing any required service patches and drivers. Refer to your hardware vendor documentation for details about this process.
Verify your installation using the Cluster Verification Utility (CVU) as in the following steps:
From the directory CRS_home\bin on the existing nodes, run the CVU command to verify your installation at the post hardware installation stage where node_list is a list of the nodes to include in the CVU operation as shown in the following example:
cluvfy stage -post hwos -n node_list|all [-verbose]
After you have configured the hardware and operating systems on the new nodes, you can use this command to verify node reachability, for example, to all of the nodes from the local node. You can also use this command to verify user equivalence to all given nodes from the local node, node connectivity among all of the given nodes, accessibility to shared storage from all of the given nodes, and so on.
|
Note: You can only use theall option with the -n argument if you have set the CV_NODE_ALL variable to represent the list of nodes on which you want to perform the CVU operation. |
From the directory CRS_home\bin on the existing nodes, run the CVU command to obtain a detailed comparison of the properties of the reference node with all of the other nodes that are part of your current cluster environment where node is the node, node_list is a list of the nodes to include, orainventory_group is the name of the Oracle inventory group, and osdba_group is the DBA group name:
cluvfy comp peer [ -refnode node ] -n node_list [-r { 10gR1 | 10gR2 } ] [ -orainv orainventory_group ] [ -osdba osdba_group ] [-verbose]
|
Note: For the reference node, select a node from your existing cluster nodes against which you want CVU to compare, for example, the newly added nodes that you specify with the-n option. |
To verify that your installation is configured correctly, perform the following steps:
Ensure that the new nodes can access the private interconnect. This interconnect must be properly configured before you can complete the procedures in "Step 2: Extending Oracle Software to New Nodes at the Oracle Clusterware Layer".
If you are not using a cluster file system, then determine the location on which your cluster software was installed on the existing nodes. Make sure that you have at least 250MB of free space on the same location on each of the new nodes to install the Oracle Clusterware. In addition, ensure you have enough free space on each new node to install the Oracle binaries.
Ensure that the Oracle Cluster Registry (OCR) and the voting disk are accessible by the new nodes using the same path as the other nodes use. In addition, the OCR and voting disk devices must have the same permissions as on the existing nodes.
Ensure that user equivalence is established on the new nodes by performing the following:
Make sure that you can run the following command from all of the existing nodes of your cluster where the hostname is the public network name of the new node:
NET USE \\hostname\C$
You have the required administrative privileges on each node if the operating system responds with:
Command completed successfully.
After completing the procedures in this section, your new nodes are connected to the cluster and configured with the required software to make them visible to the clusterware. Configure the new nodes as members of the cluster by extending the cluster software to the new nodes as described in "Step 2: Extending Oracle Software to New Nodes at the Oracle Clusterware Layer".
|
Note: Do not change a hostname after the Oracle Clusterware installation. This includes adding or deleting a domain qualification. |
The following section describes how to add new nodes to the Oracle Clusterware and to the Oracle database software layers using the OUI. The OUI requires access to the private interconnect that you checked as part of the installation validation in Step 1.
On one of the existing nodes go to the CRS home\oui\bin directory and run the addnode.bat script to start the OUI.
The OUI runs in the add node mode and the OUI Welcome page appears. Click Next and the Specify Cluster Nodes for Node Addition page appears.
Click Next and the OUI verifies connectivity on the existing nodes and on the new nodes. The verifications that the OUI performs include determining whether:
The nodes are up
The nodes are accessible by way of the network
|
Note: If any of the existing nodes are down, then you can proceed with the procedure. However, once the nodes are up, you must run the following command on each of those nodes:setup.exe -updateNodeList -local
"CLUSTER_NODES={available_node_list}"
ORACLE_HOME=CRS_home
This operation should be run from the |
The virtual hostnames are not already in use on the network
The user has write permission to create the Oracle Clusterware home on the new nodes
The user has write permission to the OUI inventory in the C:\Program Files\Oracle\Inventory directory
If the OUI detects that the new nodes do not have an inventory location, then:
The OUI automatically updates the inventory location in the Registry key
If any verifications fail, then the OUI re-displays the Specify Cluster Nodes for Node Addition page with a Status column in both tables indicating errors. Correct the errors or deselect the nodes that have errors and proceed. However, you cannot deselect existing nodes; you must correct problems on nodes that are already part of your cluster before you can proceed with node addition. If all of the checks succeed, then the OUI displays the Node Addition Summary page.
The Node Addition Summary page displays the following information showing the products that are installed in the Oracle Clusterware home that you are extending to the new nodes:
The source for the add node process, which in this case is the Oracle Clusterware home
The private node names that you entered for the new nodes
The new nodes that you entered
The required and available space on the new nodes
The installed products listing the products that are already installed on the existing the Oracle Clusterware home
Click Next and the OUI displays the Cluster Node Addition Progress page.
The Cluster Node Addition Progress page shows the status of the cluster node addition process. The table on this page has two columns showing the phase of the node addition process and the phase's status according to the following:
This page shows the following three OUI phases:
Copy the Oracle Clusterware Home to New Nodes—Copies the Oracle Clusterware home to the new nodes unless the Oracle Clusterware home is on the Oracle Cluster File System.
Perform Oracle Home Setup—Updates the Registry entries for the new nodes, creates the services, and creates folder entries.
Save Cluster Inventory—Updates the node list associated with the Oracle Clusterware home and its inventory.
The Cluster Node Addition Progress page's Status column displays In Progress while the phase is in progress, Suspended when the phase is pending execution, and Succeeded after the phase completes. On completion, click Exit to end the OUI session. After the OUI displays the End of Node Addition page, click Exit to end the OUI session.
From CRS_home\install on node1, run the crssetup.add.bat script.
Use the following command to perform an integrated validation of the Oracle Clusterware setup on all of the configured nodes, both the pre-existing nodes and the nodes that you have added
cluvfy comp stage -post crinst -n all [-verbose]
The CVU -post crinst stage check verifies the integrity of the Oracle Clusterware components. After you have completed the procedures in this section for adding nodes at the Oracle Clusterware layer, you have successfully extended the Oracle Clusterware home from your existing the Oracle Clusterware home to the new nodes. Proceed to Step 3 to prepare the storage for RAC on the new nodes.
You can optionally run addnode.bat in silent mode, replacing steps 1 through 6 as follows, where nodeI, nodeI+1, and so on are the new nodes that you are adding:
addnode.bat -silent "CLUSTER_NEW_NODES={nodeI,nodeI+1,…nodeI+n}"
"CLUSTER_NEW_PRIVATE_NODE_NAMES={node-privI,node-privI+1,…node-privI+n}"
"CLUSTER_NEW_VIRTUAL_HOSTNAMES={node-vipI,node-vipI+1,…,node-vipI+n}"
You can alternatively specify the variable=value entries in a response file and run addnode as follows:
addnode.bat -silent -responseFile filename
Command-line values always override response file values.
|
See Also: Oracle Universal Installer and OPatch User's Guide for details about how to configure command-line response files |
Run crssetup.add.bat on the local node, or on the node on which you were performing this procedure.
After you have completed the procedures in this section for adding nodes at the Oracle Clusterware layer, you have successfully extended the Oracle Clusterware home from your existing the Oracle Clusterware home to the new nodes. Proceed to "Step 3: Preparing Storage on New Nodes" to prepare storage for RAC on the new nodes.
To extend an existing RAC database to your new nodes, configure the shared storage for the new instances to be added on new nodes so that the storage type is the same as the storage that is already used by the existing nodes' instances. Prepare the same type of storage on the new nodes as you are using on the other nodes in the RAC environment that you want to extend as follows:
Automatic Storage Management (ASM)
If you are using ASM, then make sure that the new nodes can access the ASM disks with the same permissions as the existing nodes.
Oracle Cluster File System (OCFS)
If you are using Oracle Cluster File Systems, then make sure that the new nodes can access the cluster file systems in the same way that the other nodes access them.
Raw Device Storage
If your cluster database uses raw devices, then prepare the new raw devices by following the procedures described in"Raw Device Storage Preparation for New Nodes".
|
See Also: Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide for Microsoft Windows for more information about the Oracle Cluster File System |
If you are using Oracle Cluster File System, then run the following command to verify your cluster file system and obtain detailed output where node_list includes both the pre-existing nodes and the newly added nodes:
cluvfy comp ssa -n node_list -s storageID_list [-verbose]
|
See Also: Appendix A, "Troubleshooting" for more information about enabling and using the Cluster Verification Utility (CVU) |
To prepare raw device storage on the new nodes, you need at least two new disk partitions to accommodate the redo logs for each new instance. Make these disk partitions the same size as the redo log partitions that you configured for the existing nodes' instances. Also create an additional logical partition for the undo tablespace for automatic undo management. You can create a raw device mapping file and set the DBCA_RAW_CONFIG environment variable so that it points to the raw device mapping file.
|
See Also: Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide for Microsoft Windows for more information about configuring raw partitions and using raw device mapping files |
Perform the following steps from one of the existing nodes of the cluster:
Create or identify an extended partition.
Right click inside an unallocated part of the extended partition.
Choose Create from the Partition menu. A dialog box appears in which you should enter the size of the partition. Ensure you use the same sizes as those you used on your existing nodes.
Click the newly created partition and select Assign Drive Letter from the Tool menu.
Select Don't Assign Drive Letter, and click OK.
Repeat steps 2 through 5 for the second and any additional partitions.
Select Commit Changes Now from the Partition menu to save the new partition information.
Enable the new and the existing nodes to recognize all of the partitions on the cluster. To do this, create logical names so that the existing nodes and the new nodes recognize the partitions by following these steps:
Start the Object Link Manager (OLM) by entering the following command from the CRS home\bin directory on one of the existing nodes:
GUIOracleOBJManager
The OLM starts and automatically detects the logical names of the partitions and displays them in OLM's graphical user interface.
Recall the disk and partition numbers for the partitions that you created in the previous section. Look for the disk and partition numbers in the OLM page and perform the following tasks:
Right-click next to the box under the New Link column and enter the logical name for the first partition.
Repeat the previous step for the second and any additional partitions.
For example, if your RAC database name is db and your database consists of two instances running on two nodes and you are adding a third instance on the third node, then your logical names for your redo logs should be db_redo3_1, db_redo3_2, and so on.
To enable automatic undo management for a new node's instance, enter the logical name for the partition for the undo tablespace that you created in the previous section. For example, if your RAC database name is db and if it has two instances running on two nodes and you are adding a third instance on a third node, then your logical name for the undo tablespace should be db_undotbs3.
Select Commit from the Options menu. This creates the new logical names on the current node.
Select Sync Nodes from the Options menu. This makes the new logical names visible to all of the nodes in the cluster, including the new nodes.
Select Exit from the Options menu to exit the Object Link Manager.
After completing the procedures in this section, you have configured your cluster storage so that the new nodes can access the Oracle software. Additionally, the existing nodes can access the new nodes and instances.
Run the following command to verify that the prepared raw device storage is accessible from all of the configured cluster nodes where nodelist includes both the pre-existing nodes and the newly added nodes:
cluvfy comp ssa -n node_list -s storageID_list -verbose
|
See Also: Appendix A, "Troubleshooting" for more information about enabling and using the Cluster Verification Utility (CVU) |
Proceed to the next section, "Step 4: Adding Nodes at the Oracle RAC Database Layer".
To add nodes at the Oracle RAC database later, run the OUI in add node mode to configure your new nodes. If you have multiple Oracle homes, then perform the following steps for each Oracle home that you want to include on the new nodes:
On an existing node from the Oracle_home\oui\bin run the addnode.bat script. This starts the OUI in the add node mode and displays the OUI Welcome page. Click Next on the Welcome page and the OUI displays the Specify Cluster Nodes for Node Addition page.
The Specify Cluster Nodes for Node Addition page has a table showing the existing nodes associated with the Oracle home from which you launched the OUI. A node selection table appears on the bottom of this page showing the nodes that are available for addition. Select the nodes that you want to add and click Next.
The OUI verifies connectivity and performs availability checks on both the existing nodes and on the nodes that you want to add. Some of checks performed determine whether:
The nodes are up
The nodes are accessible by way of the network
The user has write permission to create the Oracle home on the new nodes
The user has write permission to the OUI inventory in the C:\Program Files\Oracle\Inventory directory on the existing nodes and on the new nodes
If the new nodes do not have an inventory set up, then the OUI automatically updates the Registry entries for the inventory location. If any of the other checks fail, then fix the problem and proceed or deselect the node that has the error and proceed. You cannot deselect existing nodes; you must correct any problems on the existing nodes before proceeding with node addition. If all of the checks succeed, then the OUI displays the Node Addition Summary page.
The Node Addition Summary page has the following information about the products that are installed in the Oracle home that you are going to extend to the new nodes:
The source for the add node process, which in this case is the Oracle home
The existing nodes and new nodes
The new nodes that you selected
The required and available space on the new nodes
The installed products listing all of the products that are already installed in the existing Oracle home
Click Finish and the OUI displays the Cluster Node Addition Progress page.
The Cluster Node Addition Progress page shows the status of the cluster node addition process. The table on this page has two columns showing the phase of the node addition process and the phase's status. The Cluster Node Addition Progress shows the following three OUI phases:
Copy the Oracle Home To New Nodes—Copies the entire Oracle home to the new nodes unless the Oracle home is on a cluster file system
Performs Oracle Home Setup—Updates the Registry entries for the new nodes, creates the services, and creates folder entries
Save Cluster Inventory—Updates the node list associated with the Oracle home and its inventory
The Cluster Node Addition Progress page's Status column displays Succeeded if the phase completes, In Progress if the phase is in progress, and Suspended when the phase is pending execution. After the OUI displays the End of Node Addition page, click Exit to end the OUI session.
Add a Listener to the new node by running the Net Configuration Assistant (NetCA) from the new node and selecting only the new node on the Node Selection page.
After completing the procedures in this section, you have defined the new nodes at the cluster database layer. You can now add database instances to the new nodes as described in Step 5.
You can optionally run addnode.bat in silent mode, replacing steps 1 through 6, as follows where nodeI, nodeI+1, and so on are the new nodes that you are adding:
addnode.bat -silent "CLUSTER_NEW_NODES={nodeI,nodeI+1,…nodeI+n}"
The variable=value entries can also be specified in a response file, filename, and addnode can be run as:
addnode.bat -silent -responseFile filename
|
See Also: Oracle Universal Installer and OPatch User's Guide for details about how to configure command-line response files |
Command-line values always override response file values. Proceed to the next section, "Step 5: Adding Database Instances to New Nodes".
You can use Enterprise Manager, or the Database Configuration Assistant (DBCA) in either interactive mode or in silent mode, to add database instances to new nodes as described under the following headings:
Using Enterprise Manager to Add Database Instances to New Nodes
Using DBCA in Interactive Mode to Add Database Instances to New Nodes
Using DBCA in Silent Mode to Add Database Instances to New Nodes
To add a database instance to a new node with Enterprise Manager, perform the following procedure:
From the Cluster Database Home page, click the Maintenance tab.
Under the Deployments section, click Add Instance. This initiates a wizard to guide you through the instance addition process.
Perform the following tasks for each wizard step:
If the database was using ASM, then provide host and ASM credentials.
Specify the hosts for which you want to add instances.
Review and submit the job.
After you submit the job, the wizard provides a summary page that shows whether the job succeeded and the wizard also provides other detailed information about the job, such as the elapsed time and an output log.
After adding the instances to the new nodes using the steps described in this section, perform any needed service configuration procedures as described in Chapter 6, "Introduction to Workload Management".
To add a database instance to a new node with DBCA in interactive mode, perform the following procedure:
Start the Database Configuration Assistant (DBCA) by entering dbca at the system prompt from the bin directory in the Oracle_home directory.
The DBCA displays the Welcome page for RAC. Click Help on any DBCA page for additional information.
Select Real Application Clusters database, click Next, and the DBCA displays the Operations page.
Select Instance Management, click Next, and the DBCA displays the Instance Management page.
Select Add Instance and click Next. The DBCA displays the List of Cluster Databases page that shows the databases and their current status, such as ACTIVE, or INACTIVE.
From the List of Cluster Databases page, select the active RAC database to which you want to add an instance. Enter user name and password for the database user that has SYSDBA privileges. Click Next and DBCA displays the List of Cluster Database Instances page showing the names of the existing instances for the RAC database that you selected.
Click Next to add a new instance and the DBCA displays the Adding an Instance page.
On the Adding an Instance page, enter the instance name in the field at the top of this page if the instance name that the DBCA provides does not match your existing instance naming scheme. Then select the new node name from the list, click Next, and the DBCA displays the Services Page.
Enter the services information for the new node's instance, click Next, and the DBCA displays the Instance Storage page.
If you are using raw devices or raw partitions, then on the Instance Storage page select the Tablespaces folder and expand it. Select the undo tablespace storage object and a dialog appears on the right-hand side. Change the default datafile name to the raw device name for the tablespace.
If you are using raw devices or raw partitions or if you want to change the default redo log group file name, then on the Instance Storage page select and expand the Redo Log Groups folder. For each redo log group number that you select, the DBCA displays another dialog box. Enter the raw device name that you created in the section "Raw Device Storage Preparation for New Nodes" in the File Name field.
If you are using a cluster file system, then click Finish on the Instance Storage page. If you are using raw devices, then repeat step 10 for all of the other redo log groups, click Finish, and the DBCA displays a Summary dialog.
Review the information on the Summary dialog and click OK or click Cancel to end the instance addition operation. The DBCA displays a progress dialog showing the DBCA performing the instance addition operation. When the DBCA completes the instance addition operation, the DBCA displays a dialog asking whether you want to perform another operation.
After you terminate your DBCA session, run the following command to verify the administrative privileges on the new node and obtain detailed information about these privileges where nodelist consists of the newly added nodes:
cluvfy comp admprv -o db_config -d oracle_home -n nodelist [-verbose]
After adding the instances to the new nodes using the steps described in this section, perform any needed service configuration procedures as described in Chapter 6, "Introduction to Workload Management".
You can use the Database Configuration Assistant (DBCA) in silent mode to add instances to nodes onto which you have extended an Oracle Clusterware home and an Oracle home. Use the following syntax to perform this operation where node is the node onto which you want to add the instance, gdbname is the global database name, instname is the name of the new instance, sysdba is the name of an Oracle user with SYSDBA privileges, and password is the password for the user name in sysdba:
dbca -silent -addInstance -nodeList node -gdbName gdbname [-instanceName instname] -sysDBAUserName sysdba -sysDBAPassword password
Note that you only need to provide an instance name if you want to override the Oracle naming convention for RAC instance names.
After you have completed either of the DBCA procedures in this section, the DBCA has successfully added the new instance to the new node and completed the following steps:
Created and started an ASM instance on each new node if the existing instances were using ASM
Created a new database instance on each new node
Created and configured high availability components
Created the Oracle Net configuration
Started the new instance
Created and started services if you entered services information on the Services Configuration page
After adding the instances to the new nodes using the steps described in this section, perform any needed service configuration procedures as described in Chapter 6, "Introduction to Workload Management".
After you add a node to a RAC database on Windows-based systems, you must manually create the following directories in the Oracle_base\Oracle_home\oc4j\j2ee\isqlplus directory before you can run iSQL*Plus on the new node:
connectors
log
persistence
tldcache
After you create these directories, you can start iSQL*Plus either by running isqlplusctl start at the command prompt or by starting iSQL*Plus from the Windows Control Panel Services tool. If you try to connect to the iSQL*Plus URL without creating these directories, then you will not be able to connect.
To add nodes to a cluster when the new nodes already have clusterware and Oracle software installed on them, you must configure the new nodes with the Oracle software that is on the existing nodes of the cluster. To do this, you must run two versions of an OUI process: one for the clusterware and one for the database layer as described in the following procedures:
Add new nodes at the Oracle Clusterware layer by running the OUI using the following command:
CRS_home\oui\bin\addnode.bat -noCopy
Add new nodes at the Oracle software layer by running the OUI from the Oracle home as follows:
Oracle_home\oui\bin\addnode.bat -noCopy
In the -noCopy mode, the OUI performs all add node operations except for the copying of software to the new nodes.
|
Note: Oracle recommends that you back up your voting disk and Oracle Cluster Registry files after you complete the node addition process. |
This section explains how to delete nodes from clusters using detailed manual procedures. The details of these steps appear in the following sections:
The procedures in this section explain how to use Enterprise Manager, or DBCA in interactive or silent mode, to delete an instance from a RAC database.
Using Enterprise Manager to Delete Database Instances from Existing Nodes
Using DBCA in Interactive Mode to Delete Database Instances from Existing Nodes
Using DBCA in Silent Mode to Delete Instance from Existing Nodes
To delete an instance with Enterprise Manager from an existing node, perform the following steps:
From the Cluster Database Home page, click the Maintenance tab.
Under the Deployments section, click Delete Instance. This action initiates a wizard to guide you through the deletion process.
To delete an instance using the DBCA in interactive mode, perform the following steps:
Start the DBCA on a node other than the node that hosts the instance that you want to delete. On the DBCA Welcome page select Oracle Real Application Clusters Database, click Next, and the DBCA displays the Operations page.
On the DBCA Operations page, select Instance Management, click Next, and the DBCA displays the Instance Management page.
On the Instance Management page, Select Delete Instance, click Next, and the DBCA displays the List of Cluster Databases page.
Select a RAC database from which to delete an instance. Enter a user name and password for the database user that has SYSDBA privileges. Click Next and the DBCA displays the List of Cluster Database Instances page. The List of Cluster Database Instances page shows the instances that are associated with the RAC database that you selected and the status of each instance.
Select an instance to delete and click Finish.
If you have services assigned to this instance, then the DBCA Services Management page appears. Use this feature to reassign services from this instance to other instances in the cluster database.
Review the information about the instance deletion operation on the Summary page and click OK. Otherwise, click Cancel to cancel the instance deletion operation. If you click OK, then the DBCA displays a Confirmation dialog.
Click OK on the Confirmation dialog to proceed with the instance deletion operation and the DBCA displays a progress dialog showing that the DBCA is performing the instance deletion operation. During this operation, the DBCA removes the instance and the instance's Oracle Net configuration. When the DBCA completes this operation, the DBCA displays a dialog asking whether you want to perform another operation.
Click No and exit the DBCA or click Yes to perform another operation. If you click Yes, then the DBCA displays the Operations page.
Use the DBCA to delete a database instance from a node as follows, where the variables are the same as those in the preceding add instance command:
dbca -silent -deleteInstance [-nodeList node] -gdbName gdbname -instanceName instname -sysDBAUserName sysdba -sysDBAPassword password
You only need to provide a node name if you are deleting an instance from a node other than the one on which you are running DBCA.
At this point, you have accomplished the following:
De-registered the selected instance from its associated Oracle Net Services listeners
Deleted the selected database instance from the instance's configured node
Removed the Oracle Net configuration
Deleted the Oracle Flexible Architecture directory structure from the instance's configured node.
Execute the following procedures to delete nodes from Oracle clusters on Windows-based systems.
Perform the following steps on a node other than the node that you want to delete:
Use the Database Configuration Assistant (DBCA) to delete the instance.
Use the Net Configuration Assistant (NETCA) to delete the Listener.
Perform the following steps on the node that you want to delete:
|
Note: You can perform some of the steps in the following procedure in silent mode as described at the end of this section. |
If the node that you are deleting has an ASM instance, then delete the ASM instance using the procedures in the following section, "Step 3: ASM Instance Clean-Up Procedures for Node Deletion".
On the that you are deleting, for example, node2, run the following command where location is the full path location of the Oracle home:
setup.exe -updateNodeList ORACLE_HOME=location
"CLUSTER_ NODES={node2}"
Depending on whether you have a shared or non-shared Oracle home, complete one of the following two procedures:
For a non-shared home, run the OUI from the Oracle home and de-install this home. You can run the setup.exe command from Oracle_home\oui\bin to start the OUI so that you can de-install the Oracle home.
For a shared home:
Run the following command from the Oracle_home\oui\bin directory:
setup.exe –detachHome ORACLE_HOME=Oracle_home
Delete the services associated with this Oracle home.
Manually update the PATH environment variable by removing the entries associated with the Oracle home that you are removing.
Manually delete any Start menu folder items associated with the deleted Oracle home.
On node1, or in the case of multiple node installations, on any node other than the node that you are deleting, run the following command from Oracle_home\oui\bin where node_list is a comma-delimited list of nodes that are to remain part of the RAC installation:
setup.exe –updateNodeList ORACLE_HOME=Oracle_home "CLUSTER_NODES={node_list}"
From CRS_home\bin on any existing node, run the following command where remote_port is the number of the remote port:
racgons remove_config node2:remote_port
You can determine the remote port by viewing the contents of the file, CRS_ home\opmn\conf\ons.config.
Run SRVCTL to stop and remove the node applications from node2. From CRS_ home\bin run the following commands:
srvctl stop nodeapps –n node2 srvctl remove nodeapps –n node2
On node1, or on any node that you are not deleting, run the following command from CRS_home\bin where node_name is the node to be deleted and node_number is the node's number as obtained from the output of the olsnodes -n command:
crssetup del –nn node_name,node_number
On the node or nodes that you are deleting, node2 in this example, run the following command from CRS_home\oui\bin:
setup.exe –updateNodeList ORACLE_HOME=CRS_home
"CLUSTER_NODES={node2}" CRS=TRUE –local
On node2, using OUI setup.exe from CRS_home\oui\bin:
If you have a non-shared home, then de-install the Oracle Clusterware installation by running the setup.exe script from CRS_home\oui\bin.
If you have a shared home, then do not perform a de-installation. Instead, perform the following steps on the node that you want to delete:
Run the following command from CRS_home\oui\bin to detach the Oracle Clusterware home on node2:
setup.exe –detachHome ORACLE_HOME=CRS_home
On node2, stop any services that are associated with this Oracle Clusterware home. The associated service s are namely: OracleCRSService, OracleEVMService, OracleCSService, and OracleClusterVolumeService.
Remove the Oracle services.
Manually delete all of the Start menu items associated with this Oracle Clusterware home.
Manually update the PATH environment variable to remove entries associated with this Clusterware home.
Then remove the following files under %SystemRoot%\System32\drivers: ocfs.sys, orafencedrv.sys, and orafenceservice.sys.
On node1, or in the case of a multiple node installation, on any node other than the one to be deleted, run the following command from CRS_home\oui\bin where node_list is a comma-delimited list of nodes that are to remain part of the RAC database:
setup.exe –updateNodeList ORACLE_HOME=CRS_home "CLUSTER_NODES={node_list}" CRS=TRUE
As mentioned at the beginning of this procedure, you can optionally delete nodes from Real Application Clusters databases in silent mode by completing the following steps:
Perform steps 1 and 2 from the previous procedure.
On the node to be deleted, node2 in this case:
If you have a non-shared home, then de-install the Oracle home as follows:
setup.exe –silent -deinstall "REMOVE_HOMES={Oracle_home}"
If the home is a shared home, then do not perform a de-install. Instead, perform the following steps on the node to be deleted:
Perform a detach home on node2 by running the following command from Oracle_home\oui\bin on node2:
setup.exe –detachHome –silent ORACLE_HOME=Oracle_home
On node 2, stop and delete any services that are associated with this Oracle home. In addition, delete any Registry entries and path entries that are associated with this Oracle home. Also, delete all of the start menu items associated with this Oracle home.
Perform steps 4 through 8 from the previous procedure.
On node2, using OUI setup.exe from CRS_home\oui\bin:
If you have a non-shared home, then de-install the Oracle Clusterware home as follows:
setup.exe –silent -deinstall "REMOVE_HOMES={CRS_home}"
If you have a shared home, then do not perform a de-installation. Instead, perform the following steps on the node that you want to delete:
Run the following command from CRS_home\oui\bin to detach the Oracle Clusterware home on node2:
setup.exe –detachHome –silent ORACLE_HOME=CRS_home
On node2, stop any services that are associated with this Oracle Clusterware home. The associated service s are namely: OracleCRSService, OracleEVMService, OracleCSService, and OracleClusterVolumeService.
Remove the Oracle services.
Manually delete all of the Start menu items associated with this Oracle Clusterware home.
Manually update the PATH environment variable to remove entries associated with this Clusterware home.
Then remove the following files under %SystemRoot%\System32\drivers: ocfs.sys, orafencedrv.sys, and orafenceservice.sys.
Perform step 10 from the previous procedure.
|
Note: Oracle recommends that you back up your voting disk and Oracle Cluster Registry files after you complete the node deletion process. |
If you are using ASM, then the delete node procedure requires the following additional steps to remove the ASM instances:
If this is the Oracle home from which the node-specific Listener named LISTENER_nodename runs, then use NETCA to remove this Listener. If necessary, re-create this Listener in another home.
If this is the Oracle home from which the ASM instance runs, then remove the ASM configuration by running the following command for all nodes on which this Oracle home exists:
srvctl stop asm -n node
Then run the following command for the nodes that you are removing:
srvctl remove asm -n node
Run the following command on each node that you are going to delete that has an ASM instance:
oradim -delete -asmsid +ASMnode_number
If you are not using a cluster file system for your ASM Oracle home, then run the following commands on the deleted node:
rd /s /q ORACLE_BASE\admin\+ASM del Oracle_home\database\*ASM*
