Managing Storage Checkpoints

Use the Storage Checkpoint Administration menu option to create, display, mount, unmount, and remove Storage Checkpoints. A Storage Checkpoint can be considered an online database backup that contains a point-in-time database image. Storage Checkpoints can later be used to restore the image of a file, a tablespace, or the entire database to any earlier state recorded by the Storage Checkpoints.

This operation displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

VxDBA uses the repository to determine the list of tablespaces, datafiles, and file systems for Storage Checkpoint creation and removal. Select from the following Storage Checkpoint Administration operations:

Use this menu option to create a Storage Checkpoint as a point-in-time image of the database and to back up control file, initialization file, and log information. The database can be offline or online when you create a Storage Checkpoint.

Use this menu option to display the Storage Checkpoints created by VxDBA. After displaying the VxDBA Storage Checkpoints, VxDBA asks if you want to display any other Storage Checkpoints, such as Storage Checkpoints created by the Capacity Planning Utility or NetBackup.

Use this menu option to mount a Storage Checkpoint into the file system namespace. Mounted Storage Checkpoints appear as any other file system on the machine, and you can access mounted Storage Checkpoints using all normal file system-based commands.

Use this menu option to unmount a mounted Storage Checkpoint.

Use this menu option to remove Storage Checkpoints that you no longer need.

Creating Storage Checkpoints

This operation creates a Storage Checkpoint (point-in-time image) of the database instance. A Storage Checkpoint for a database is a collection of Storage Checkpoints created at the same time on each of the file systems where the database files reside.

It is recommended that you take a Storage Checkpoint after you convert to or from Quick I/O files.

The database can be offline or online when a Storage Checkpoint is created. If the database is online when the Storage Checkpoint is created, VxDBA switches the database to online backup mode before creating the Storage Checkpoint. Once the Storage Checkpoint is created, VxDBA switches the database back to its normal operation mode.

In addition to creating a Storage Checkpoint, VxDBA also automatically backs up the associated control files, initialization file, and log information. Suppose that you made a structural change to your database, and then needed to roll back the database to a Storage Checkpoint that was created before the structural change. The Storage Rollback would only be successful if you could also reconstruct the database to the same structure that it was when the Storage Checkpoint was created. You can recreate the previous database structure using the control files, initialization file, and log information that were backed up when the Storage Checkpoint was created.

Create New Storage Checkpoints displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Click the thumbnail above to view full-sized image.

Displaying Storage Checkpoints

This operation displays the list of Storage Checkpoints created by the VxDBA utility on the file systems used by the Oracle instance. VxDBA also asks if you want to display Storage Checkpoints created by other applications, such as Capacity Planning Utility or NetBackup.

While you see and manage only a single Storage Checkpoint, the Storage Checkpoint is a collection of Storage Checkpoints. Each file system used by the database contains a Storage Checkpoint of the same name, and it is this collection of Storage Checkpoints across file systems that you manage as a single Storage Checkpoint in VxDBA.

Storage Checkpoints are of the following types:

• ON–Online
• OF–Offline
• IN–Instant
• UN–Unknown

Storage Checkpoints may have additional status appended to show other attributes. Current additional status modifiers are:

• M–Mounted
• R–Read-only
• W–Writable
• I–Invalid for current Oracle SID
• C–Complete
• P–Partial

The status of a Storage Checkpoint can be 'C' for Complete or 'P' for partial. Complete means that the Storage Checkpoint successfully completed across all file systems used by the database, and each file system contains the same Storage Checkpoint name. Partial means that the Storage Checkpoint operation did not successfully complete or that one or more of the file systems used by the database does not contain the named Storage Checkpoint. Partial Storage Checkpoints can happen if, for example:

• You are in the middle of creating a new Storage Checkpoint and the database or system crashes. When the system or database is back online, VxDBA detects that not all of the file systems used by the database contain the named Storage Checkpoint.

You should consider deleting partial Storage Checkpoints that are a result of a database or system crash, and create a new Storage Checkpoint.

• After taking a successful, complete Storage Checkpoint, you add a tablespace on a new file system or a file system that was not previously used by the database. Again, VxDBA detects that not all of the file systems used by the database contain the named Storage Checkpoint. In this case, you can use the partial Storage Checkpoint for Storage Rollback, but only datafiles on file systems that contain complete Storage Checkpoints can be rolled back.

Be sure you understand the ramifications of rolling back to such a Storage Checkpoint (for example, losing the new tablespace). See the Oracle Backup and Recovery Guide for information and tips on restoring databases from an old backup.

• One of the file systems used by your database runs out of space and VxFS automatically removes the oldest Storage Checkpoint on that file system. VxDBA detects that the Storage Checkpoint on that file system is missing and marks the Storage Checkpoint as partial. Here again, you may be able to use the partial Storage Checkpoint for Storage Rollback, but do consider the ramifications of doing so.

To avoid this situation, use VxDBA's Monitoring Agent to monitor file system space usage on all file systems used by the database and allow VxDBA to automatically grow the file systems when they are running out of space. See Managing File System Space for more information.

• One or more of the file systems used by the database are not VxFS file systems and, therefore, do not support Storage Checkpoints. VxDBA detects that one or more file system Storage Checkpoints are missing and marks the Storage Checkpoint as partial.

Avoid using mixed file systems in support of databases when possible.

Mounted means that the Storage Checkpoint is currently mounted. Writable means that the Storage Checkpoint has been modified, by fsck or by being mounted as read-write, and is not suitable for Storage Rollback operations.

Ordering of the displayed Storage Checkpoints is by default from most recently created to least recently created. This ordering may not be intuitive, especially if you want to keep the existing list items in the same order. If you need to modify this ordering, set the VXDBA_CKPT_SORT environment variable. The default ordering for sorting Storage Checkpoint names is "-r" (most to least recent). By setting this variable to another sort option, the Status field identifies if the Storage Checkpoint is partial (P), complete (C), invalid (I), mounted (M), read-only (R), or writable (W).

Display Storage Checkpoints displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Mounting Storage Checkpoints

This operation lets you mount a Storage Checkpoint. You can mount, access, and write to Storage Checkpoints just as you can any file system. See Using Storage Checkpoints and Storage Rollback for Backup and Restore for more information.

Mount Storage Checkpoints displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Click the thumbnail above to view full-sized image.

Unmounting Storage Checkpoints

This operation lets you unmount a previously mounted Storage Checkpoint.

Unmount Storage Checkpoints displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Removing Storage Checkpoints

This operation removes Storage Checkpoints that are no longer needed. For example, you can remove a Storage Checkpoint on a file system to free up needed space.

Remove Storage Checkpoints displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Managing Storage Rollback

Use the Storage Rollback Administration menu to roll back a database file, a list of database files, a single tablespace, or the entire database to a Storage Checkpoint.

This operation displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Storage Checkpoints can only be used to roll back files that are damaged due to a software error or a human error (for example, accidental deletion of a table). Because Storage Checkpoints reside on the same physical disks as the primary file system, when a file is corrupted due to a media failure, the file on the Storage Checkpoints will not be available either. In this case, you need to restore files from a tape backup.

After the files are rolled back, you may need to follow the recovery procedure described in the Oracle manuals to recover the database before the database can be used.

Rolling Back the Database to a Storage Checkpoint

This operation rolls back the entire database (all the datafiles used by the database, except the redo logs and control files) to a Storage Checkpoint. You must shut down the database to roll back the database.

Roll Back the Database to a Storage Checkpoint displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Click the thumbnail above to view full-sized image.

VxDBA first displays a list of Storage Checkpoints for selection. After a Storage Checkpoint is selected, VxDBA rolls back every database file in parallel. Parallel Storage Rollback significantly reduces the time required to roll back a database.

Rolling Back a Tablespace to a Storage Checkpoint

If a tablespace is corrupted or removed due to a software error or a human mistake, this operation rolls back all of the files of the tablespace to a Storage Checkpoint.

  If the database is offline

Roll Back a Tablespace to a Storage Checkpoint displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

VxDBA prompts you for a tablespace name, and then displays a list of Storage Checkpoints:

Click the thumbnail above to view full-sized image.

After you select a Storage Checkpoint, VxDBA rolls back the files of the selected tablespace in parallel. Parallel Storage Rollback significantly reduces the time required to roll back a tablespace.

  If the database is online

Roll Back a Tablespace to a Storage Checkpoint displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

VxDBA prompts you for a tablespace name, and then displays a list of Storage Checkpoints:

Click the thumbnail above to view full-sized image.

After you select a Storage Checkpoint, VxDBA rolls back the files of the selected tablespace in parallel. Parallel Storage Rollback significantly reduces the time required to roll back a tablespace.

Rolling Back Datafiles to a Storage Checkpoint

This operation rolls back database files to a Storage Checkpoint. You can also use this operation to roll back more than one tablespace. Specify the list of files for Storage Rollback in a list file, or enter the list of files one by one. If the database is offline Roll Back Files to a Storage Checkpoint displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Click the thumbnail above to view full-sized image.

After you select a Storage Checkpoint, VxDBA rolls back the files to the selected Storage Checkpoint in parallel. Parallel Storage Rollback significantly reduces the time required to roll back files. When all the files are rolled back, follow the Oracle recovery procedure to recover the database before the database is used.

  If the database is online

Roll Back Files to a Storage Checkpoint displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Click the thumbnail above to view full-sized image.

After you select a Storage Checkpoint, VxDBA rolls back the files to the selected Storage Checkpoint in parallel. Parallel Storage Rollback significantly reduces the time required to roll back files. When all the files are rolled back, follow the Oracle recovery procedure to recover the database before the database is used.

Setting the Number of Storage Rollback Threads

This operation lets you configure the number of threads used when rolling back an Oracle datafile. Performance is a critical factor when rolling a file back to a Storage Checkpoint. By default, 8 lightweight threads are used to partition up and recover a datafile. Depending on the number of CPUs available on your system and the type of volume on which the file system is located, this default setting may specify too few or too many threads.

Set Number of Storage Rollback Threads displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

This menu option lets you experiment with various settings to achieve optimal performance for your system. You do not need to change the default number of threads if the Storage Rollback performance on your system is satisfactory. The maximum number of threads you can set is 63, and the minimum number of threads is 1.

Setting the Buffer Size for Storage Rollback

This operation lets you configure the buffer size used for Storage Rollback. As with setting the number of Storage Rollback threads, the buffer size configured for reads and writes when rolling back an Oracle datafile can also affect performance. By default, a 128K read/write buffer is used.

Set Buffer Size for Storage Rollback displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

This menu option lets you experiment with various settings to gain optimal performance for your system. You do not need to change the default buffer size if the Storage Rollback performance on your system is satisfactory. Set the buffer size in bytes---the minimum buffer size is 1K (1024 bytes), and the maximum buffer size is 1 MB (1,048,576 bytes).

Showing the Backup Control File List

This operation displays the list of control files that VxDBA has backed up each time you create a Storage Checkpoint using VxDBA.

Show Backup Control File List displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Following the location of the backup control file, Show Backup Control File List displays the following information:

Shows the status of the Oracle instance (Y for ONLINE or N for OFFLINE) when the Storage Checkpoint was taken. When a Storage Checkpoint is created while the Oracle instance is up, the control files, initialization file, and the log information are backed up at the same time. However, if a Storage Checkpoint is created while the Oracle instance is down, the log information will not be available. If this backed up information is removed from the Storage Checkpoint directory, the status of the INIT&CNTL? field becomes N (for null).

Shows whether the same Storage Checkpoint still exists across all file systems used by the database (see the discussion of partial Storage Checkpoints in Displaying Storage Checkpoints for more information). When one or more of the file systems used by the database does not contain the named Storage Checkpoint, the status of CKPT? becomes n (for null).

Shows whether the Oracle instance initialization file and the backup control file still exist in the Storage Checkpoint directory. If this backed up information is removed from the Storage Checkpoint directory, the status of the INIT& CNTL? field becomes N (for null).

Managing Space Usage and the VxDBA Monitoring Agent

Use the Monitoring Agent Administration menu to:

• Manage and monitor VxFS file system, Oracle tablespace, and datafile space usage
• Configure the Monitoring Agent options and statistics collection
• Start and stop the Monitoring Agent

This operation displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Managing File System Space

The VxDBA Monitoring Agent monitors the file system space, and when the space usage reaches a configured threshold value, a predefined action script grows the file system automatically. The agent can be enabled or disabled to start at boot-time. Each file system monitored has three settings that the Monitoring Agent needs to know about:

• Warning Threshold is a percent value (% of file system utilized) that determines when the agent begins warning the administrator of space shortage
• Grow Threshold is a percent value (% of file system utilized) that determines when the agent is to attempt to grow the file system (when space usage is at a critical level)
• Amount is either a percentage or a value in megabytes by which to grow file systems when the Grow Threshold is reached or exceeded

The VxDBA Monitoring Agent operations are driven from the following files:

• /opt/VRTSdbed/lib/dbed_mon_config.base
• /etc/vx/vxdba/$ORACLE_SID/dbed_mon_config.$ORACLE_SID
• /etc/vx/vxdba/$ORACLE_SID/dbed_mon_fslist.$ORACLE_SID
• /etc/vx/vxdba/$ORACLE_SID/dbed_mon_oralist.$ORACLE_SID
• /etc/vx/vxdba/$ORACLE_SID/include

The /opt/VRTSdbed/lib/dbed_mon_config.base file contains the site-level configuration settings for monitoring all file systems and databases recognized. This configuration file specifies how often to check for file system and database configuration changes, how often to check the file space usage, where space usage information gets logged, and the thresholds for warning and automatically growing the file system. By default, the monitoring agent log file is located under the /var/log/dbed_mon directory.

During file system and database configuration, the dbed_mon_config.base file gets copied into each database-specific directory as the file /etc/vx/vxdba/ORACLE_SID 
dbed_mon_config.$ORACLE_SID. For example, if you are monitoring a database named PROD, the database-specific file would be /etc/vx/vxdba/PROD/dbed_mon_config.PROD. This is the first file opened when the agent is started and contains the default settings for monitoring file systems at the database level. The VxDBA Monitoring Agent cannot start without this file. Modify this configuration file if you want to change the preconfigured settings carried over from the dbed_mon_config.base file to maintain a different set of settings at the database level.

The following two files:

• /etc/vx/vxdba/ORACLE_SID/dbed_mon_fslist.$ORACLE_SID
• /etc/vx/vxdba/$ORACLE_SID/dbed_mon_oralist.$ORACLE_SID

are created by the VxDBA utility and are used for restarting the VxDBA Monitoring Agent. These files specify the status of the database. The files also specify the space monitoring and alarm information for each file system, tablespace, and datafile. You can edit these files manually to change settings, and then use VxDBA to restart the Monitoring Agent.

The VxDBA Monitoring Agent uses the /etc/vx/vxdba/$ORACLE_SID/include file to check that all files are up-to-date and are being monitored. This file is created by the VxDBA utility and should not be edited.

Occasionally, Monitoring Agents ignore Storage Checkpoints. This happens when a Storage Checkpoint is not owned by the current Oracle instance. These Storage Checkpoints will not be used to calculate thresholds and potential removal candidates. Storage Checkpoints that are not considered part of the current Oracle database instance's data set are logged as such in the file /var/log/dbed_mon/dbed_mon.prune_ckpt_log.$ORACLE_SID when the Monitoring Agent is looking for potential removal candidates. A Storage Checkpoint must have an entry in the /etc/vx/vxdba/$ORACLE_SID/checkpoint_dir directory before it is considered owned by the database. This is done automatically by the provided VxDBA and dbed_ckpt create utilities and ensures that, if multiple databases share the same file system(s), the policy for one database does not affect another.

Use the File System Space Administration menu to monitor the space usage of the file system and each Storage Checkpoint. You can also use the menu to enable or disable the VxDBA Monitoring Agent.

This operation displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Select from the following File System Space Alarm Administration operations:

Display File System Space Usage

Use this menu option to display file system space usage information and the estimated space used for Storage Checkpoints created by VxDBA.

Display File System Space Alarm Settings

Use this menu option to display the space alarm information on the file systems used by the Oracle instance. The menu operation provides the boot-time status and current status of the VxDBA Monitoring Agent, as well as the list of file systems with their associated space alarm settings and status (ENABLED or DISABLED).

Enable/Disable/Modify Space Alarm Settings

Use this menu option to control the Monitoring Agent activity and modify the current space alarm settings on the file systems used by the Oracle instance.

Displaying File System Space Usage

This operation displays the space usage of the file system and the Storage Checkpoints used by the Oracle instance. For example, you can use this operation to monitor the daily database change history and use this data for capacity planning to forecast the additional disk space needed for Storage Checkpoints.

Display File System Usage displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

In this example, the space used by Checkpoint_901426272 is less than 1 MB, which means the Storage Checkpoint does not contain many data blocks. This means that the database has not modified many distinct data blocks since this Storage Checkpoint was created. Another Storage Checkpoint may have been created after this one, with all subsequent changes going to the new Storage Checkpoint.

Displaying File System Space Alarm Settings

This operation displays the information about the space alarm settings defined for the file systems used by the Oracle instance.

The space alarm relies on the VxDBA Monitoring Agent. The agent daemon processes must be running first. If the agent daemons are not running, a message is displayed asking you to start the agent daemons.

Click the thumbnail above to view full-sized image.

After you start the VxDBA Monitoring agent, Display File System Space Alarm Settings displays the list of file systems and the space alarm status:

Click the thumbnail above to view full-sized image.

Enabling, Disabling, or Modifying Space Alarm Settings

When the file system runs out of space, VxFS automatically removes Storage Checkpoints to free up space. This could happen when Oracle is processing update transactions such that original data blocks are saved in the Storage Checkpoints. Enabling the space alarm allows VxDBA to monitor space usage and grow the file systems automatically, so that Storage Checkpoints are not unnecessarily removed.

The Enable/Disable/Modify Space Alarm Settings operation first checks to see if you are logged in as root. If you are not logged in as root, VxDBA prompts you for the root password:

Click the thumbnail above to view full-sized image.

After you enter the root password, Enable/Disable/Modify Space Alarm Settings displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Select from the following file system space alarm operations:

Use this menu option to change the boot-time start activity of the VxDBA Monitoring Agent. You are provided with the current setting (ENABLED or DISABLED), and then prompted for changes.

Use this menu option to configure the monitoring and expansion policy for all file systems. You are prompted for the Warning Threshold for space usage, the Grow Threshold for space usage, and the Amount as a percentage or a value in megabytes by which to grow the file system. These three policy values are then used for all file systems unless a per file system policy is set.

Use this menu option to configure the monitoring and expansion policy for a particular file system. After displaying the current policy per file system, you are asked if you want to change the policies and are then prompted for the new values for the Warning Threshold for space usage, the Grow Threshold for space usage, and the Amount as a percentage or a value in megabytes by which to grow the file system. If you want to disable monitoring of a particular file system, set the Grow Threshold and the amount to Grow By values to zero. By default, the expansion of file systems is disabled and must be enabled by the user.

Use this menu option to re-read the /etc/vx/vxdba/$ORACLE_SID/dbed_mon_fslist.$ORACLE_SID file. Select this operation if you manually edit this file.

Use this menu option to start or stop the monitoring agent. After displaying the current status of the Monitoring Agent (running or not running), you can either start or stop the Monitoring Agent.

Managing Oracle Tablespace and Datafile Space

Use the Oracle Tablespace/DatafileSpace Administration menu to monitor the space usage of Oracle tablespaces and datafiles, and to display or modify the VxDBA Monitoring Agent's Oracle space alarm settings. You can also use the menu to enable or disable the VxDBA Monitoring Agent.

This operation displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Select from the following File System Space Alarm Administration operations:

Use this menu option to display datafile space usage information. Total size of Oracle objects, free space available, and Oracle blocks are displayed. Oracle objects and free space available are given in MB.

Use this menu operation to display the space alarm information on the Oracle tablespaces. The menu display provides the boot-time and current status of the VxDBA Monitoring Agent and the list of Oracle tablespaces with their associated space alarm settings and status (ENABLED or DISABLED).

Use this menu option to change the boot-time start activity of the VxDBA Monitoring Agent. You are provided with the current setting (ENABLED or DISABLED), and then prompted for changes.

Displaying Oracle Tablespace/Datafile Space Usage

This operation displays the space usage of Oracle tablespaces, datafiles, and the Storage Checkpoints used by the Oracle instance.

Display Oracle Tablespace/Datafile Space Usage displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Displaying Oracle Tablespace/Datafile Space Alarm Settings

This operation displays the information about the space alarm settings defined for the Oracle tablespaces and datafiles.

The space alarm relies on the VxDBA Monitoring Agent. The agent daemon processes must be running first. If the agent daemons are not running, a message is displayed asking you to start the agent daemons.

The expansion of Oracle Tablespaces is not currently supported. The display for settings on the grow threshold and the grow amount always shows N/A.

Click the thumbnail above to view full-sized image.

Once you start the VxDBA Monitoring agent, Display Oracle Tablespace/Datafile / Space Alarm Settings displays the list of tablespaces and datafiles and the space alarm status:

Click the thumbnail above to view full-sized image.

Enabling, Disabling, or Modifying Oracle Space Alarm Settings

Enabling the Oracle space alarm allows VxDBA to monitor tablespace and datafile space usage. The warning is sent to the log file

/var/log/dbed_mon/dbed_mon.logfile.$ORACLE_SID 

The Enable/Disable/Modify Oracle Space Alarm Settings operation first checks to see if you are logged in as root. If you are not logged in as root, VxDBA prompts you for the root password:

Click the thumbnail above to view full-sized image.

After you enter the root password, Enable/Disable/Modify Space Alarm Settings displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Select from the following Oracle space alarm operations:

Use this menu option to change the boot-time start activity of the monitoring agent. The message provides you with the current setting and prompts you for changes.

Use this menu option to configure the monitoring policy for all Oracle tablespaces. The menu prompts you for the Warning Threshold for space usage.

Use this menu option to configure the monitoring policy for a particular Oracle tablespace. After displaying the current policy per Oracle tablespace, the program asks if you want to change the policies and then prompts you for the new values for the Warning Threshold.

Use this menu option to re-read the /etc/vx/vxdba/$ORACLE_SID/dbed_mon_oralist.$ORACLE_SID  file. Select this operation if you manually edited this file.

Use this menu option to start or stop the monitoring agent.

Configuring Monitoring Agent Options

Use this menu operation to modify current default settings for the Monitoring Agent. The agent configuration is saved under the following:

/etc/vx/vxdba/$ORACLE_SID/dbed_mon_config.$ORACLE_SID

When the configuration file is modified, restart the Monitoring Agent for the changes to take effect. This operation displays a screen similar to the following:

Click the thumbnail above to view full-sized image.

Click the thumbnail above to view full-sized image.

Click the thumbnail above to view full-sized image.