B Overview of Real Application Clusters

Overview of the Cluster Database Home Page

You log in to the RAC database Oracle Enterprise Manager Console from a client browser, as described in the section "Starting and Stopping the Oracle Enterprise Manager Console" in Chapter 3, "Getting Started with Oracle Enterprise Manager".

When you log in to the Oracle Enterprise Manager Console, the Cluster Database Home Page for the RAC database opens.

The Cluster Database Home page looks similar to a single-instance Database Control Home Page. However, on the Cluster Database Home Page, Oracle Enterprise Manager displays the system state and availability of the entire RAC database.

In the General Heading section, a traffic signal light image shows the state of the RAC database. If the database is up, then there is a green arrow pointing up in the image. Also, next to the General section, there is a Shutdown button. If the database is down, then there is a red arrow pointing down in the image, and there is a Startup button. Red x mark icons indicate system states that need attention; green check mark icons indicate system states that are within normal expectations. You can also see detailed metric displays of the RAC database state by clicking the Performance tab.

The Alerts table is displayed under the metrics section on the Cluster Database Home Page. It lists clusterwide database alerts in order of severity, and sorted by database components. For more information about the alert, click the text in the Message column. This takes you to the specific page that measures the system state of the process that triggered the alert.

On the Cluster Database Home Page level, you can select the Performance, Administration, and Maintenance tabs to perform clusterwide administration tasks. You can obtain assistance in understanding the options for each task, or the meanings for each metric, by clicking the Help link next to the Database tab. After you navigate to individual metrics displays, when you want to return to the Cluster Database Home Page, click the Database tab in the top right corner of the page.

Viewing RAC Database Targets

On the Cluster Database Page, a target is a cluster, a database instance, or an Oracle utility, such as a listener. The console obtains metrics that provide information about the current state of targets. These metrics are presented in separate section headings for each target. On the Cluster Database Cluster Home Page, Oracle Enterprise Manager shows high-level metrics about the state of the cluster. By clicking the names of individual targets on the page, you can navigate to more detailed pages that display metrics about the target you have selected.

You can review all targets in the RAC database by navigating to the cluster database Targets page. To view all targets, complete the following procedure:

1. Log in to Oracle Enterprise Manager Database Control.

2. On the Cluster Database Home Page, in the General heading section, click the Cluster name link.

3. On the Cluster Home Page, click the Targets tab.

   The Targets page provides an overview of the current state of each target. By clicking the name of the target, you can navigate to detailed metrics pages for each target. For database and ASM instances, clicking the target name opens the Home Page for that target, so that you can monitor individual database states.

Navigating to Oracle Enterprise Manager Cluster-Related Pages

You can monitor various activities in your cluster database environment with the other pages in Oracle Enterprise Manager described in this section.

The Cluster Database Performance page, which you access by clicking the Performance tab from the Cluster Database Home page, a quick glimpse of the performance statistics for a database. Statistics are rolled up across all of the instances in a RAC database. Using links that are next to the charts at the bottom of the page you can drill down to:

• Identify the causes of performance issues

• Decide whether resources need to be added or redistributed

• Tune your SQL plan and schema for better optimization

• Resolve performance issues

For performance information about individual instances, you can use the Cluster Database Instance Performance page. To reach this page for a particular instance, click the Performance tab on the Cluster Database Home page, then click the name of the instance you want to monitor name at the bottom of the page. You can drill down from links beside the charts or at the bottom of the Database Instance Performance page to do the following:

• Identify the cause of performance issues

• Decide whether resources need to be added or redistributed to resolve performance issues

• Tune your SQL plan and schema for better optimization

• Resolve performance issues

Use the Cluster Performance Page to see utilization statistics, such as CPU, Memory, and Disk reads and writes, during the past hour for all nodes of your cluster. You access the Cluster Performance page by clicking the Cluster link in the General section of the Cluster Database Performance page, then clicking the Performance tab from the Cluster Home page. With this information, you can determine whether resources need to be added, suspended, or redistributed. From this page, you can view the CPU, Memory, and Disk I/O charts for each node individually by clicking the host name in the legend to the right of the charts. The resulting Host page also shows statistics for CPU load, memory scan page rate, and longest service time.

The Cluster Interconnects page enables you to monitor the interconnect interfaces, determine configuration issues, and identify transfer rate-related issues including excess traffic. This page helps determine the load added by individual instances and databases on the interconnect. Sometimes you can immediately identify interconnect delays that are due to applications that are outside of the Oracle database. To access Cluster Interconnects page, click the Cluster link in the General section of the Cluster Database Home page, then click the Interconnects tab. The tables on this page enable you to do the following:

• View all interfaces that are configured across the cluster

• View statistics for the interfaces, such as absolute transfer rates and errors

• Determine the type of interfaces, such as private or public

• Determine whether the instance is using a public or private network

• Determine which database instance is currently using which interface

• Determine how much the instance is contributing to the transfer rate on the interface

Viewing and Modifying RAC Database Initialization Parameters

Managing initialization parameters across the RAC database is essentially the same as managing them for a single-instance database. Note the following differences for parameters in a RAC database:

• Parameters that are cluster-specific parameters are identified in the Category column with the designation Cluster Database

• Parameters that are duplicated on each instance in the RAC database are identified in the Instance column with an asterisk (*)

• Parameters that can be set differently on each instance of the RAC database are listed by instance number

Note that RAC database parameters are different from single-instance database parameters. If you change a parameter setting marked by an asterisk, or designated as a Cluster Database parameter, then this changes that setting for each instance of your RAC database. If you change a numbered instance parameter, that is, a parameter for one instance of the RAC Database, it does not change parameter settings on other database instances in the RAC database.

Starting and Stopping RAC Databases

The following instructions describe the process of starting and stopping the entire RAC database, including each cluster database instance:

1. Log in to Oracle Enterprise Manager Database Control

2. On the Cluster Database page, and in the General view, click Startup (if the database is down) or Shutdown (if the database is up)

3. On the Specify Credentials page, enter the cluster database host credentials for the database nodes. The host credentials are the username and password for a user who is a member of the OSDBA or OSOPER system privileges group.

4. On the Startup/Shutdown: Select Operation page, click Startup All to start all instances, or click Shutdown All to close all instances

5. On the Startup/Shutdown: Confirmation page, click Yes

Starting and Stopping Individual Instances in a RAC Database

The following instructions describe how to navigate to a single instance on a RAC database, and start or stop it:

1. Log in to Oracle Enterprise Manager Database Control

2. On the Cluster Database Home Page, in the General heading section, click the Cluster name link

3. On the Cluster:name Home Page (where name is the name of the cluster), click the Targets tab

4. On the Cluster:name Target page (where name is the name of the cluster), in the Name column of the targets table, click the name of the instance that you want to start up or shut down.

5. On the Cluster Database Instance: name page (where name is the name of the instance), click Startup (if the database is down) or Shutdown (if the database is up)

6. On the Startup/Shutdown: Specify Host and Target Database Credentials page, provide host (OSDBA) credentials and database (SYSDBA) credentials, and click OK

7. On the Startup/Shutdown: Confirmation page, click Yes

Redo Log Groups and Redo Threads in Real Application Cluster Databases

Redo logs contain a record of changes that have been made to datafiles. In a single-instance database environment, redo logs generated by database datafile changes are stored in two or more redo log file groups. Each of these groups contains a redo log file, and, possibly, one or more mirrored copies of that file. In a RAC database, each instance requires its own set of redo log groups, known as its "thread" of redo. Mirrored copies of the redo log files provide your system with an extra layer of protection against hardware failures or data corruption.

Each redo log thread must contain at least two redo log groups. Oracle recommends that each of your instances has a redo thread containing the same number of redo log groups and, as with single-instance databases, each group should contain the same number of members. For example, in a RAC database with two instances, each instance could have a thread of redo containing five redo log groups, for a total of ten redo log groups for the database. Each of these groups could contain two members: a redo log and its mirrored copy. If you create the RAC database with DBCA, then it automatically implements a configuration that meets Oracle recommendations.

In a RAC database, each instance writes and archives the redo log groups in its own thread in the same manner single-instance databases do, as described in Chapter 6, "Managing Database Storage Structures". However, when in recovery mode, the instance performing the recovery is able to read and process all of the redo threads for the database, no matter which instance generated them. This enables a running instance to recover the work completed by a failed instance, so that end users can continue with their interrupted work without waiting for the failed instance to be restarted. For example, assume that you have a RAC database with two instances, instance A and instance B. If instance A is down, then instance B can read the redo log files both for instance A and B to ensure successful recovery.

In a RAC database using ASM, all redo log files are stored on shared storage, and each instance has transparent access to the redo log files for all other instances in the cluster.

To access the redo log file groups:

1. Go to the Cluster Database Home Page, and click the Administration tab

2. On the Cluster Database Administration page, under the Storage column, select Redo Log Groups

At the Redo Log Groups page, you can add redo log groups and add members to the redo log group. The Thread column identifies the instance thread to which a redo log file belongs

Backing Up RAC Databases

When you use Automatic Storage Management (ASM) to manage database files, Oracle recommends that you use Recovery Manager (RMAN) for backup.

To back up a RAC database:

1. Go to the Cluster Database Home Page, and click the Maintenance tab

2. On the Cluster Database Maintenance page, under the Backup/Recovery column, select Schedule Backup

3. Follow the backup procedures outlined in Chapter 9, "Performing Backup and Recovery"

Overview of Restore and Recovery Operations for a RAC Database

This section discusses both instance recovery and media recovery.

Instance recovery is automatic. It can take place while the database is running. If a RAC instance fails, then a surviving database instance processes the online redo logs generated by the failed instance to ensure the database contents are in a consistent state. With the addition of more nodes, the performance of the RAC database during automatic instance recovery becomes progressively faster. When recovery is complete, Oracle clusterware attempts to restart the failed instance automatically.

Media recovery is a manual process that takes place while the database is closed, typically to recover and restore database files after catastrophic disk failures, block corruption, or user errors. Media recovery is always done by one instance in the cluster.

With RMAN, the process of recovering and restoring a RAC database is essentially the same as for a single-instance database, except that you access RMAN from the Maintenance page at the cluster database level, instead of the instance level.

To recover a RAC database:

1. Go to the Cluster Database Home Page, and click the Maintenance tab

2. On the Cluster Database Maintenance page, under the Backup/Recovery column, select Perform Recovery

3. Follow the recovery procedures outlined in "Performing Restore and Recovery Operations" in Chapter 9, "Performing Backup and Recovery"

Using a Flash Recovery Area with a RAC Database

To use a flash recovery area with a RAC database, the recovery area must be shared among all instances. To accomplish this, Oracle recommends that you place the flash recovery area on the shared ASM disks. In addition, the parameters DB_RECOVERY_FILE_DEST and DB_RECOVERY_FILE_DEST_SIZE must be set to the same values on all instances.

To use Flash Recovery, you must first configure the flash recovery area, as described in "Configuring the Flash Recovery Area" in Chapter 9, "Performing Backup and Recovery".