Moving Disk Groups Between Systems
An important feature of disk groups is that they can be moved between systems. If all disks in a disk group are moved from one system to another, then the disk group can be used by the second system. You do not have to re-specify the configuration.
To move a disk group between systems, use the following procedure:
-
On the first system, stop all volumes in the disk group, then deport (disable local access to) the disk group with the following command:
# vxdg deport diskgroup
-
Move all the disks to the second system and perform the steps necessary (system-dependent) for the second system and VxVM to recognize the new disks.
This can require a reboot, in which case the vxconfigd daemon is restarted and recognizes the new disks. If you do not reboot, use the command vxdctl enable to restart the vxconfigd program so VxVM also recognizes the disks.
-
Import (enable local access to) the disk group on the second system with this command:
# vxdg import diskgroup
 Caution
All disks in the disk group must be moved to the other system. If they are not moved, the import fails.
-
After the disk group is imported, start all volumes in the disk group with this command:
# vxrecover -g diskgroup -sb
You can also move disks from a system that has crashed. In this case, you cannot deport the disk group from the first system. When a disk group is created or imported on a system, that system writes a lock on all disks in the disk group.
Caution
The purpose of the lock is to ensure that dual-ported disks (disks that can be accessed simultaneously by two systems) are not used by both systems at the same time. If two systems try to access the same disks at the same time, this must be managed using software such as the clustering functionality of VxVM. Otherwise, configuration information stored on the disk may be corrupted, and the data on the disk may become unusable.
Handling Errors when Importing Disks
When you move disks from a system that has crashed or that failed to detect the group before the disk was moved, the locks stored on the disks remain and must be cleared. The system returns the following error message:
VxVM vxdg ERROR V-5-1-587 disk group groupname: import failed: Disk is in use by another host
The next message indicates that the disk group does not contains any valid disks (not that it does not contains any disks):
VxVM vxdg ERROR V-5-1-587 Disk group groupname: import failed:
No valid disk found containing disk group
The disks may be considered invalid due to a mismatch between the host ID in their configuration copies and that stored in the /etc/vx/volboot file.
To clear locks on a specific set of devices, use the following command:
# vxdisk clearimport devicename ...
To clear the locks during import, use the following command:
# vxdg -C import diskgroup
Caution
Be careful when using the vxdisk clearimport or vxdg -C import command on systems that have dual-ported disks. Clearing the locks allows those disks to be accessed at the same time from multiple hosts and can result in corrupted data.
You may want to import a disk group when some disks are not available. The import operation fails if some disks for the disk group cannot be found among the disk drives attached to the system. When the import operation fails, one of several error messages is displayed.
The following message indicates a fatal error that requires hardware repair or the creation of a new disk group, and recovery of the disk group configuration and data:
VxVM vxdg ERROR V-5-1-587 Disk group groupname: import failed: Disk group has no valid configuration copies
The following message indicates a recoverable error.
VxVM vxdg ERROR V-5-1-587 Disk group groupname: import failed: Disk for disk group not found
If some of the disks in the disk group have failed, force the disk group to be imported with the command:
# vxdg -f import diskgroup
Caution
Be careful when using the -f option. It can cause the same disk group to be imported twice from different sets of disks, causing the disk group to become inconsistent.
These operations can also be performed using the vxdiskadm utility. To deport a disk group using vxdiskadm, select menu item 8 (Remove access to (deport) a disk group). To import a disk group, select item 7 (Enable access to (import) a disk group). The vxdiskadm import operation checks for host import locks and prompts to see if you want to clear any that are found. It also starts volumes in the disk group.
Reserving Minor Numbers for Disk Groups
A device minor number uniquely identifies some characteristic of a device to the device driver that controls that device. It is often used to identify some characteristic mode of an individual device, or to identify separate devices that are all under the control of a single controller. VxVM assigns unique device minor numbers to each object (volume, plex, subdisk, disk, or disk group) that it controls.
When you move a disk group between systems, it is possible for the minor numbers that it used on its previous system to coincide (or collide) with those of objects known to VxVM on the new system. To get around this potential problem, you can allocate separate ranges of minor numbers for each disk group. VxVM uses the specified range of minor numbers when it creates volume objects from the disks in the disk group. This guarantees that each volume has the same minor number across reboots or reconfigurations. Disk groups may then be moved between machines without causing device number collisions.
VxVM chooses minor device numbers for objects created from this disk group starting at the base minor number base_minor. Minor numbers can range from this value up to 16,777,215. Try to leave a reasonable number of unallocated minor numbers near the top of this range to allow for temporary device number remapping in the event that a device minor number collision may still occur.
VxVM reserves the range of minor numbers from 0 to 999 for use with volumes in the boot disk group. For example, the rootvol volume is always assigned minor number 0.
If you do not specify the base of the minor number range for a disk group, VxVM chooses one at random. The number chosen is at least 1000, is a multiple of 1000, and yields a usable range of 1000 device numbers. The chosen number also does not overlap within a range of 1000 of any currently imported disk groups, and it does not overlap any currently allocated volume device numbers.
Note
The default policy ensures that a small number of disk groups can be merged successfully between a set of machines. However, where disk groups are merged automatically using failover mechanisms, select ranges that avoid overlap.
To view the base minor number for an existing disk group, use the vxprint command as shown in the following examples for the disk group, mydg:
# vxprint -l mydg | egrep minors
minors: >=45000
# vxprint -g mydg -m | egrep base_minor
base_minor=45000
To set a base volume device minor number for a disk group that is being created, use the following command:
# vxdg init diskgroup minor=base_minor disk_access_name ...
For example, the following command creates the disk group, newdg, that includes the specified disks, and has a base minor number of 30000:
# xvdg init newdg minor=30000 c1d0t0 c1t1d0
If a disk group already exists, you can use the vxdg reminor command to change its base minor number:
# vxdg -g diskgroup reminor new_base_minor
For example, the following command changes the base minor number to 30000 for the disk group, mydg:
# vxprint -g mydg reminor 30000
If a volume is open, its old device number remains in effect until the system is rebooted or until the disk group is deported and re-imported. If you close the open volume, you can run vxdg reminor again to allow the renumbering to take effect without rebooting or re-importing.
An example of where it is necessary to change the base minor number is for a cluster-shareable disk group. The volumes in a shared disk group must have the same minor number on all the nodes. If there is a conflict between the minor numbers when a node attempts to join the cluster, the join fails. You can use the reminor operation on the nodes that are in the cluster to resolve the conflict. In a cluster where more than one node is joined, use a base minor number which does not conflict on any node.
For further information on minor number reservation, see the vxdg(1M) manual page.
Compatibility of Disk Groups Between Platforms
For disk groups that support the Cross-platform Data Sharing (CDS) feature, the upper limit on the minor number range is restricted on AIX, HP-UX, Linux (with a 2.6 or later kernel) and Solaris to 65,535 to ensure portability between these operating systems.
On a Linux platform with a pre-2.6 kernel, the number of minor numbers per major number is limited to 256 with a base of 0. This has the effect of limiting the number of volumes and disks that can be supported system-wide to a smaller value than is allowed on other operating system platforms. The number of disks that are supported by a pre-2.6 Linux kernel is typically limited to a few hundred. With the extended major numbering scheme that was implemented in VxVM 4.0 on Linux, a maximum of 4079 volumes could be configured, provided that a contiguous block of 15 extended major numbers was available.
VxVM 4.1 runs on a 2.6 version Linux kernel, which increases the number of minor devices that are configurable from 256 to 65,536 per major device number. This allows a large number of volumes and disk devices to be configured on a system. The theoretical limit on the number of DMP and volume devices that can be configured on such a system are 65,536 and 1,048,576 respectively. However, in practice, the number of VxVM devices that can be configured in a single disk group is limited by the size of the private region.
When a CDS-compatible disk group is imported on a Linux system with a pre-2.6 kernel, VxVM attempts to reassign the minor numbers of the volumes, and fails if this is not possible.
To help ensure that a CDS-compatible disk group is portable between operating systems, including Linux with a pre-2.6 kernel, use the following command to set the maxdev attribute on the disk group:
# vxdg -g diskgroup set maxdev=4079
Note
Such a disk group may still not be importable by VxVM 4.0 on Linux with a pre-2.6 kernel if it would increase the number of minor numbers on the system that are assigned to volumes to more than 4079, or if the number of available extended major numbers is smaller than 15.
You can use the following command to discover the maximum number of volumes that are supported by VxVM on a Linux host:
# cat /proc/sys/vxvm/vxio/vol_max_volumes
4079
See the vxdg(1M) manual page for more information.
|
