Setting Up Shared Storage and I/O Fencing for SFRAC

The shared storage for VERITAS Storage Foundation for Oracle RAC must support SCSI-3 persistent reservations to enable I/O fencing (see I/O Fencing). SFRAC involves two types of shared storage: data disks to store shared data, and coordinator disks, which are small LUNs (typically three per cluster), to control access to data disks by the nodes.

Setting up shared storage and I/O fencing involve the following tasks:

• Adding disks. See Adding and Initializing Disks.
• Testing data disks. See Testing Disks Using vxfentsthdw.
• Setting up coordinator disks. See Setting Up Coordinator Disks.
• Editing the VCS configuration file for I/O fencing. See Editing the UseFence Attribute in VCS Configuration.
• Stopping and starting each node to bring up the cluster configuration with I/O fencing enabled. See Starting VCS, CVM, and CFS on All Nodes.

Adding and Initializing Disks

After you physically add shared disks to the nodes, you must initialize them as VxVM disks. Use the following examples; see the VERITAS Volume Manager Administrator's Guide for more information on adding and configuring disks.

1. Use the ioscan and insf commands on each node to make the new disks recognizable:
   # ioscan -C disk
   # insf -e

   ---

    Caution    The HP-UX man page for the insf command instructs you to run the command in single-user mode only. Running insf -e in multi-user mode is acceptable assuming no other user is accessing any of the device files. This command can change the mode, owner, or group of an existing special (device) file, or unlink and recreate a file. Special files that are currently open may be left in an indeterminate state.

   ---

2. Verify that the Array Support Library (ASL) for the disk array is installed on each of the nodes. Run the following command on each node and examine the output to verify the installation of ASL:
     # vxddladm listsupport all
     LIB_NAME                  ARRAY_TYPE   VID          PID
     ============================================================== 
     .
     libvxemc.sl               A/A          EMC          SYMMETRIX
     .

   If the ASL for the array you are adding is not installed, obtain and install it on each node before proceeding. The ASL for the supported storage device you are adding is available from the disk array vendor or VERITAS technical support.

3. Scan all disk drives and their attributes, update the VxVM device list, and reconfigure DMP with the new devices. Type:
     # vxdisk scandisks
4. To initialize the disks as VxVM disks, use one of the following methods:
   • Use the interactive vxdiskadm utility to initialize the disks as VxVM disks.
   • Use the vxdisksetup command to initialize a disk as a VxVM disk. The example that follows specifies the CDS format:
   
       vxdisksetup -i device_name format=cdsdisk

   For example:

   
       # vxdisksetup -i c2t0d2 format=cdsdisk

Verifying the Nodes See the Same Disk

To confirm whether a disk (or LUN) supports SCSI-3 persistent reservations, two nodes must simultaneously have access to the same disks. Because a shared disk is likely to have a different name on each node, check the serial number to verify the identity of the disk. Use the vxfenadm command with the -i option to verify that the same serial number for the LUN is returned on all paths to the LUN.

For example, an EMC array is accessible by the /dev/rdsk/c2t13d0 path on node A and by the /dev/rdsk/c2t11d0 path on node B. From node A, enter:

The same serial number information should appear when you enter the equivalent command on node B using the /dev/rrdsk/c2t11d0 path.

On a disk from another manufacturer, Hitachi Data Systems, the output is different and may resemble:

Refer to the vxfenadm(1M) manual page.