How to replace a failed, mirrored boot disk in rootvg
The failed physical volume, hdisk0, and its mirror, hdisk1, are part of the rootvg volume group.
1. To remove mirror copies from the failed PV, run the following command:
unmirrorvg rootvg hdisk0
2. Remove hdisk0 from the boot list by running the following command:
Note: If your configuration uses boot devices other than hdisk0 and hdisk1, add them to the command syntax.
bootlist -om normal hdisk1
This ensures that hdisk1 remains a bootable device in rootvg. The absence of hdisk0 in the above command ensures that hdisk0 is no longer a bootable device.
After completing this step, ensure that hdisk0 does not appear in output by running the following command.
bootlist -m normal -o
3. Recreate any dedicated dump devices from the failed PV.
If you have a dedicated dump device that was on the failed PV, you can use the mklv command to create a new logical volume on an existing PV. Use the sysdumpdev command to set the new logical volume as the primary dump device.
4. To undefine the failed PV, run the following command:
Note: Removing the disk device entry will also remove the /dev/ipldevice hard link if the failed PV is the PV used to boot the system.
reducevg rootvg hdisk0
rmdev -dl hdisk0
5. If the failed PV is the most recently used boot device, recreate the /dev/ipldevice hard link that was removed in Step 4 by running the following command:
ln /dev/rhdisk1 /dev/ipldevice
Note the r prefixed to the PV name.
To verify that your /dev/ipldevice hard link has been recreated, run the following command:ls /dev/ipldevice
6. Replace the failed disk.
7. To define the new PV, run the following command:
cfgmgr
The cfgmgr command assigns a PV name to the replacement PV. The assigned PV name is likely to be the same as the PV name previously assigned to the failed PV. In this example, assume that the device hdisk0 is assigned to the replacement PV.
8. To add the new PV to the volume group, run the following command:
extendvg rootvg hdisk0
You might encounter the following error message:
0516-050 Not enough descriptor space left in this volume group.
Either try adding a smaller PV or use another volume group.
If you encounter this error and cannot add the PV to the volume group, you can try to mirror logical volumes to another PV that already exists in the volume group or add a smaller PV. If neither option is possible, you can try to bypass this limitation by upgrading the volume group to a Big-type or Scalable-type volume group using the chvg command.
9. Mirror the volume group.
Note: The mirrorvg command cannot be used if all of the following conditions exist:
* The target system is a logical partition (LPAR).
* A copy of the boot logical volume (by default, hd5) resides on the failed PV.
* The replacement PV's adapter was dynamically configured into the LPAR since the last cold boot.
If all of the above conditions exist, use the mklvcopy command to recreate mirror copies for each logical volume as follows:
1. Create copies of the boot logical volume to ensure that it is allocated to a contiguous series of physical partitions.
2. Create copies of the remaining logical volumes, and synchronize the copies using the syncvg command.
3. Make the disk bootable by shutting down the LPAR and activating it instead of rebooting using the shutdown or reboot commands. This shutdown does not have to be done immediately, but it is necessary for the system to boot from the new PV.
Otherwise, create new copies of logical volumes in the volume group using the new PV with the following command:
Note: The mirrorvg command disables quorum by default.
For rootvg, you will want to use the -m option to ensure that the new logical volume copies are mapped to hdisk0 in the same way as the working disk.mirrorvg -m rootvg hdisk0
10. If your configuration holds third copies of some logical volumes, you might need to recreate those copies with the following command:
mklvcopy -k
11. Initialize the boot record by running the following command:
bosboot -a
12. If the PV failure occurred on rootvg, update the boot list by running the following command:
Note: If your configuration uses boot devices other than hdisk0 and hdisk1, add them to the command.
bootlist -om normal hdisk0 hdisk1
13. Verify that the procedure was successful.
* To verify that all logical volumes are mirrored to the new PV, run the following command:
lslv lvname
Check the COPIES attribute of each logical volume affected by the failed PV to ensure that the desired number of copies now exist. If the number of copies of the logical volume is below the desired number, use the mklvcopy command to create additional copies.
* To verify that all the logical volume partitions are synchronized, check that there are no stale partitions by running the following command:
lspv hdisk0
Check the STALE PARTITIONS attribute of the replaced PV to ensure that the count is zero. If there are stale partitions use the syncvg command to synchronize the partitions.
Use the following steps to verify other aspects of this procedure:
* To verify the boot list, run the following command:
bootlist -om normal
* To verify the dump device, run the following command:
sysdumpdev -l
* To verify the list of bootable PVs, run the following command:
ipl_varyon -i
* To verify the /dev/ipl_device, run the following command:
ls -i /dev/rhdisk1 /dev/ipldevice
Ensure the output of the ls command has the same i-node number for both entries.
This step completes the procedure.