3 Grid Control Common Configurations

3.5.1.1 Monitoring the Load on Your Management Service Installations

To keep the workload evenly distributed, you should always be aware of how many Management Agents are configured for each Management Service and monitor the load on each Management Service.

At any time, you can view a list of Management Agents and Management Services using the Setup tab of the Grid Control Console.

Use the charts on the Overview page of the Management Services and Repository tab to monitor:

• Loader backlog (files)

  The Loader is part of the Management Service that pushes metric data into the repository at periodic intervals. If the Loader Backlog chart indicates that the backlog is high and Loader output is low, there is data pending load, which may indicate a system bottleneck or the need for another Management Service. The chart shows the total backlog of files totalled over all Oracle Management Services for the past 24 hours. Click the image to display loader backlog charts for each individual Management Service over the past 24 hours.

• Notification delivery backlog

  The Notification Delivery Backlog chart displays the number of notifications to be delivered that could not be processed in the time allocated. Notifications are delivered by the Management Services. This number is summed across all Management Services and is sampled every 10 minutes. The graph displays the data for the last 24 hours. It is useful for determining a growing backlog of notifications. When this graph shows constant growth over the past 24 hours, then you may want to consider adding another Management Service, reducing the number of notification rules, and verifying that all rules and notification methods are useful and valid.

3.5.1.2 Monitoring the Response Time of the Enterprise Manager Web Application Target

The information on the Management Services and Repository tab can help you determine the load being placed on your Management Service installations. More importantly, you should also consider how the performance of your Management Service installations is affecting the performance of the Grid Control Console.

Use the EM Website Web Application target to review the response time of the Grid Control Console pages:

1. From the Grid Control Console, click the Targets tab and then click the Web Applications subtab.

2. Click EM Website in the list of Web Application targets.

3. In the Key Test Summary table, click homepage. The resulting page provides the response time of the Grid Control Console homepage URL.

   
   

   See Also: The Enterprise Manager online help for more information about using the homepage URL and Application Performance Management (also known as Application Performance Monitoring) to determine the performance of your Web Applications

   
   

4. Click Page Performance to view the response time of some selected Grid Control Console pages.

   
   

   Note: The Page Performance page provides data generated only by users who access the Grid Control Console by way of the OracleAS Web Cache port (usually, 7777).

   
   

5. Select 7 Days or 31 Days from the View Data menu to determine whether or not there are any trends in the performance of your Grid Control installation.

   Consider adding additional Management Service installations if the response time of all pages is increasing over time or if the response time is unusually high for specific popular pages within the Grid Control Console.

3.6.1.1 Configuring the Management Services for High Availability

The Management Service for Grid Control 10g Release 2 has a new high availability feature called the Shared Filesystem Loader. In the Shared Filesystem Loader, management data files received from Management Agents are stored temporarily on a common shared location called the shared receive directory. All Management Services are configured to use the same storage location for the shared receive directory. The Management Services coordinate internally and distribute amongst themselves the workload of uploading files into the Management Repository. Should a Management Service go down for some reason, its workload is taken up by surviving Management Services.

Configuring the Shared Filesystem Loader

To configure the Management Service to use Shared Filesystem Loader, you must use the emctl config oms loader command.

1. Stop all the Management Services.

2. Configure a shared receive directory that is accessible by all Management Services.

3. Run emctl config oms loader -shared yes -dir <loader directory> individually on all Management Services hosts, where <loader directory> is the full path to the shared receive directory.

4. Restart all Management Services.

3.6.1.2 Understanding the Flow of Data When Load Balancing the Upload of Management Data

Figure 3-4 shows a typical scenario where a set of Management Agents upload their data to a server load balancer, which redirects the data to one of two Management Service installations.

Figure 3-4 Load Balancing Between the Management Agent and the Management Service

Description of "Figure 3-4 Load Balancing Between the Management Agent and the Management Service"

In this example, only the upload of Management Agent data is routed through the server load balancer. The Grid Control Console still connects directly to a single Management Service by way of the unique Management Service upload URL.

When you load balance the upload of Management Agent data to multiple Management Service installations, the data is directed along the following paths:

1. Administrators can use one of two URLs to access the Grid Control Console just as they do in the multiple Management Service configuration defined in Section 3.5.

2. Each Management Agent uploads its data to a common server load balancer URL. This URL is defined in the emd.properties file for each Management Agent. In other words, the Management Agents connect to a virtual service exposed by the server load balancer. The server load balancer routes the request to any one of a number of available servers that provide the requested service.

3. Each Management Service, upon receipt of data, stores it temporarily in a local file and acknowledges receipt to the Management Agent. The Management Services then coordinate amongst themselves and one of them loads the data in a background thread in the correct chronological order.

4. Each Management Service communicates by way of JDBC with a common Management Repository, just as they do in the multiple Management Service configuration defined in Section 3.5.

5. Each Management Service communicates directly with each Management Agent by way of HTTP, just as they do in the multiple Management Service configuration defined in Section 3.5.

3.6.1.3 Configuring a Server Load Balancer for Management Agent Data Upload

This section describes some guidelines for configuring a server load balancer to balance the upload of data from Management Agents to multiple Management Service installations.

Specifically, you should use the administration tools that are packaged with your server load balancer to configure a virtual pool that consists of the hosts and the services that each host provides. In the case of the Management Services pool, specify the host name and agent upload port. To insure a highly available Management Service, you should have two or more Management Services defined within the virtual pool.

Modify the REPOSITORY_URL property in the emd.properties file located in the sysman/config directory of the Management Agent home directory. The host name and port specified must be that of the server load balancer virtual service.

Declare the pool to use a load balancing policy, for example, Round Robin or Least Loaded. Do not configure persistence between Management Agents and Management Services.

This configuration allows the load balancer to distribute connections from Management Agents equally between Management Services. In the event a Management Service becomes unavailable, the load balancer should be configured to direct connections to the surviving Management Services.

To successfully implement this configuration, the load balancer can be configured to monitor the underlying Management Service. On some models, for example, you can configure a monitor on the server load balancer. The monitor defines the:

• HTTP request that is to be sent to a Management Service

• Expected result in the event of success

• Frequency of evaluation

For example, the load balancer can be configured to check the state of the Management Service every 5 seconds. On three successive failures, the load balancer can then mark the component as unavailable and no longer route requests to it. The monitor should be configured to send the string GET /em/upload over HTTP and expect to get the response Http XML File receiver.

3.6.2.1 Understanding the Flow of Data When Load Balancing the Grid Control Console

Figure 3-5 shows a typical configuration where a server load balancer is used between the Management Agents and multiple Management Services, as well as between the Grid Control Console and multiple Management Services.

Figure 3-5 Load Balancing Between the Grid Control Console and the Management Service

Description of "Figure 3-5 Load Balancing Between the Grid Control Console and the Management Service"

In this example, a single server load balancer is used for the upload of data from the Management Agents and for the connections between the Grid Control Console and the Management Service.

When you use a server load balancer for the Grid Control Console, the management data uses the following paths through the Grid Control architecture:

1. Administrators use one URL to access the Grid Control Console. This URL directs the browser to the server load balancer virtual service. The virtual service redirects the browser to one of the Management Service installations. Depending upon the host name and port selected by the server load balancer from the virtual pool of Management Service installations, the Grid Control Console obtains the management data by way of OracleAS Web Cache and the Oracle HTTP Server on one of the Management Service hosts.

2. Each Management Agent uploads its data to a common server load balancer URL (as described in Section 3.6.1) and data is written to the shared receive directory.

3. Each Management Service communicates by way of JDBC with a common Management Repository, just as they do in the multiple Management Service configuration defined in Section 3.5.

4. Each Management Service communicates directly with each Management Agent by way of HTTP, just as they do in the multiple Management Service configuration defined in Section 3.5.

3.6.2.2 Configuring a Server Load Balancer for the Grid Control Console

Use the administration tools that are packaged with your server load balancer to configure a virtual pool that consists of the hosts and the services that each host provides. In the case of the Management Services pool, specify the host name and default OracleAS Web Cache port. To insure a highly available Management Service, you should have two or more Management Services defined within the virtual pool.

The load balancer parcels the work to any number of Management Service processes that it has in its virtual pool. This provides a method for constant communication to the Grid Control Console in the event of the failure of a Management Service.

The virtual pool for Grid Control Console should to be configured for session persistence. It is necessary that all requests from one user go to the same Management Service for the duration of a session. Use the persistence method provided by your load balancer. For example if you have enabled Enterprise Manager Framework Security and you are running the Management Service in a secure environment (using HTTPS and SSL), use SSL Session ID based persistence. If you have not enabled Enterprise Manager Framework Security and you are running in an environment that is not secure (using HTTP), you could use Client IP or Cookie based persistence.

3.6.2.3 Configuring Oracle HTTP Server When Using a Server Load Balancer for the Grid Control Console

The Management Service is implemented as a J2EE Web application, which is deployed on an instance of Oracle Application Server. Like many Web-based applications, the Management Service often redirects the client browser to a specific set of HTML pages, such as a logon screen and a specific application component or feature.

When the Oracle HTTP Server redirects a URL, it sends the URL, including the Oracle HTTP Server host name, back to the client browser. The browser then uses that URL, which includes the Oracle HTTP Server host name, to reconnect to the Oracle HTTP Server. As a result, the client browser attempts to connect directly to the Management Service host and bypasses the server load balancer.

To prevent the browser from bypassing the load balancer when a URL is redirected, edit the ServerName directive defined in the Oracle HTTP Server configuration file. This directive will be found in one of two places:

• If you have enabled Enterprise Manager Framework Security and you are running the Management Service in a secure environment (using HTTPS and SSL), the ServerName directive you must change is located in the following configuration file:

  ORACLE_HOME/Apache/Apache/conf/ssl.conf
  
  

• If you have not enabled Enterprise Manager Framework Security and you are running in an environment that is not secure (using HTTP), the ServerName directive you must change is located in the following configuration file:

  ORACLE_HOME/Apache/Apache/conf/httpd.conf
  
  

Change the ServerName directive so it matches the name of the server load balancer virtual service that you configured in Section 3.6.2.2.

3.6.3.1 Understanding the Flow of Data When Configuring the Management Repository for High Availability

Figure 3-6 shows a typical Grid Control high availability configuration, where server load balancers are balancing the load on the multiple Management Service installations and the Management Repository is protected by Oracle Real Application Clusters and Oracle Data Guard.

Figure 3-6 Grid Control High Availability Configuration

Description of "Figure 3-6 Grid Control High Availability Configuration"

When you install the Management Repository in a RAC database and incorporate Oracle Data Guard into the configuration, the management data uses the following paths through the Grid Control architecture:

1. Administrators use one URL to access the Grid Control Console. This URL directs the browser to the server load balancer virtual service as described in Section 3.6.2.

2. Each Management Agent uploads its data to a common server load balancer URL as described in Section 3.6.1.

   
   

   Caution: Before deploying a server load balancer for the upload of Management Agent data, be sure to review Section 3.6.1.3, "Configuring a Server Load Balancer for Management Agent Data Upload"

   
   

3. Each Management Service communicates by way of JDBC with a common Management Repository, which is installed in a Real Application Clusters instance. Each Management Service uses the same database connection information, defined in the emoms.properties file, to load data into the Management Repository. The Management Service uses the same connection information to pull data from the Management Repository as it is requested by the Grid Control Console.

   
   

   See Also: "Configuring the Management Service to Use Oracle Net Load Balancing and Failover" for information about configuring the connection to a Management Repository that is installed in a RAC database

   
   

   In addition, the Management Repository is also protected by Oracle Data Guard. Note that only physical Data Guard is supported for protecting the Management Repository.

   
   

   See Also: Oracle Data Guard Concepts and Administration

   
   

4. Each Management Service communicates directly with each Management Agent by way of HTTP, just as they do in the multiple Management Service configuration defined in Section 3.5.

3.6.3.2 Installing the Management Repository into a Real Applications Clusters (RAC) Database

To install the Management Repository into a RAC database, use the following procedure:

1. Install the Oracle 10g Database Release 2 (10.2) software and create a RAC database.

2. Begin installing Grid Control, using the Enterprise Manager 10g Grid Control Using an Existing Database installation option.

3. When you are prompted for a database system identifier (SID) and port, specify the SERVICE_NAME for one of the RAC instances.

4. After the Grid Control installation is complete, modify the Management Service connection string to take advantage of client failover in the event of a RAC host outage.

   
   

   See Also: "Configuring the Management Service to Use Oracle Net Load Balancing and Failover"

   
   

5. The Management Services rely on repository database listener's connect time load balancing to distribute connections between RAC instances. For the distribution to work optimally in Enterprise Manager Grid Control, ensure that PREFER LEAST LOADED NODE <listener_name> property in listener.ora files is commented out or set to ON.

3.6.3.3 Specifying the Size of the Management Repository Tablespaces in a RAC Database

When you install the Management Repository into a RAC database instance, you cannot set the size of the required Enterprise Manager tablespaces. You can, however, specify the name and location of data files to be used by the Management Repository schema. The default sizes for the initial data file extents depend on using the AUTOEXTEND feature and as such are insufficient for a production installation. This is particularly problematic when storage for the RAC database is on a raw device.

If the RAC database being used for the Management Repository is configured with raw devices, there are two options for increasing the size of the repository.

• You can create multiple raw partitions, with the first one equal to the default size of the tablespace as defined by the installation process.

• Alternatively, you can create the tablespace using the default size, create a dummy object that will increase the size of the tablespace to the end of the raw partition, then drop that object.

Regardless, if raw devices are used, disable the default space management for these objects, which is to auto-extend.

An alternative for RAC installations is to use ASM managed storage. The location string will have to be specified manually (for example, +DATA/<database_name>/<datafile_name). If ASM storage is used, there is no need to disable any space management storage settings.

3.6.3.4 Configuring the Management Service to Use Oracle Net Load Balancing and Failover

When you use a RAC cluster, a standby system, or both to provide high availability for the Management Repository, the Management Service can be configured to use an Oracle Net connect string that will take advantage of redundancy in the repository. Correctly configured, the Management Service process will continue to process data from Management Agents even during a database node outage.

To configure the Management Service to take advantage of this feature:

1. Use a text editor to open the following configuration file in the Management Service home directory:

   ORACLE_HOME/sysman/config/emoms.properties
   
   

2. Locate the following entry in the emoms.properties file:

   oracle.sysman.eml.mntr.emdRepConnectDescriptor=
   
   

3. Edit the entry so it includes references to the individual nodes within the RAC database.

   The following example shows a connect string that supports a two-node RAC configuration. Note the backslash (\) before each equal sign (=), which is required when you are entering the connect string within the emoms.properties configuration file:

   oracle.sysman.eml.mntr.emdRepConnectDescriptor=(DESCRIPTION\=(ADDRESS_
   LIST\=(FAILOVER\=ON)(ADDRESS\=(PROTOCOL\=TCP)(HOST\=haem1.us.oracle.com)(PORT\
   1521))(ADDRESS\=(PROTOCOL\=TCP)(HOST\=haem2.us.oracle.com)(PORT\=1521)))
   (CONNECT_DATA\=(SERVICE_NAME\=em10)))
   

   
   

   See Also: "Enabling Advanced Features of Oracle Net Services" in the Oracle Database Net Services Administrator's Guide for more information about using the FAILOVER parameter and other advanced features within a database connect string