oracledba.help
SpecialTopics

UDev Configuration

Overview

UDev guarantees persistent device names across reboots regardless of their order of discovery. This is vital for ASM storage devices. UDev also ensures the devices have the correct ownership for ASM. The following covers configuring Oracle Linux UDev to manage disks for ASM.

Oracle fully supports the Linux 8\7 methods detailed here using UDev via SYMLINK method as opposed to the Linux 6.x legacy method of using the mapper architecture (per 3-17731438021).

ASMLib has been deprecated in favor of UDev or ASMFD.

  • Udev is not required for standard disks. To configure\mount a standard disk using its UUID go here.
  • If you are using VMWare you may need to do this to display the UUID.

Linux 8 Procedure

The scenario below shows implementing Udev with 3 shared disks.

Confirm Hardware Detected

 /sbin/partprobe
 for host in /sys/class/scsi_host/*; do echo "- - -" | sudo tee $host/scan; ls /dev/sd* ; done
 lsblk

Add Option to scsi_id.config

 echo options=-g>>/etc/scsi_id.config

This ensures the SCSI devices are configured as trusted.

Get Device ID

 udevadm info --query=all --name=/dev/sdb | egrep "WWN|SERIAL"
 Example: ID_SERIAL=36000c29654b6aa069e1553a3251c4f56
  • Note the value of ID_SERIAL.
  • Do this for each disk (sdb,sdc,sdd...).

Create 96-asm.rules

Create\Edit: /etc/udev/rules.d/96-asm.rules

For each disk:

  • Set ENV{ID_SERIAL} to the ID_SERIAL string from above.
  • Set\increment the SYMLINK name for each disk.
 KERNEL=="sd*1",
 SUBSYSTEM=="block",
 ENV{ID_SERIAL}=="36000c29654b6aa069e1553a3251c4f56",
 SYMLINK+="asm-disk01",
 OWNER="grid",
 GROUP="asmadmin",
 MODE="0660"

The above goes on one line.

3 Disk Template

 KERNEL=="sd*", SUBSYSTEM=="block", ENV{ID_SERIAL}=="TBD", SYMLINK+="asm-disk01", OWNER="grid", GROUP="asmadmin", MODE="0660"
 KERNEL=="sd*", SUBSYSTEM=="block", ENV{ID_SERIAL}=="TBD", SYMLINK+="asm-disk02", OWNER="grid", GROUP="asmadmin", MODE="0660"
 KERNEL=="sd*", SUBSYSTEM=="block", ENV{ID_SERIAL}=="TBD", SYMLINK+="asm-disk03", OWNER="grid", GROUP="asmadmin", MODE="0660"

Replace TBD with matching ID_SERIAL value.

UDev Check

Ensure udev shows bound to correct disk and UUID.

 udevadm test /block/sdb/sdb1|more
 ...
 DEVNAME=/dev/sdb1
 ID_SERIAL=36000c29654b6aa069e1553a3251c4f56

Do this for each disk (sdb,sdc,sdd...).

Reload the udev/rules

 udevadm control --reload-rules
 udevadm trigger --type=devices --action=change

Test

Ensure asm-disknn bound to correct Linux volume.

 ls -al /dev/asm*
 lrwxrwxrwx 1 root root 4 Oct 14 13:05 /dev/asm-disk01 -> sdb1
 lrwxrwxrwx 1 root root 4 Oct 14 13:05 /dev/asm-disk02 -> sdc1
 lrwxrwxrwx 1 root root 4 Oct 14 13:05 /dev/asm-disk03 -> sdd1

Ensure devices devices owned by grid:asmadmin.

 ls -rlt /dev/sd?1
 brw-rw---- 1 grid asmadmin 8, 49 Oct 14 13:05 /dev/sdd1
 brw-rw---- 1 grid asmadmin 8, 17 Oct 14 13:05 /dev/sdb1
 brw-rw---- 1 grid asmadmin 8, 33 Oct 14 13:05 /dev/sdc1

They may not appear in sequential order. That is OK.

Next Node

Repeat the above procedure for all other RAC node systems. Use the Device ID as displayed from first system.

Reboot

After configuring UDev it is a good idea to reboot all nodes and ensure all the disks look OK via:

  1. Disable CRS autostart: crsctl disable crs
    Important Note: It is wise to disable CRS autostart as ASM can become
    corrupted if it tries to start and the disks are not bound correctly.
  2. Reboot
  3. Test (as shown above)
ls -al /dev/asm*
ls -rlt /dev/sd?1
  1. Start CRS if everything looks OK.

Linux 7 Procedure

The scenario below shows implementing Udev with 3 shared disks.

Confirm Hardware Detected

 /sbin/partprobe
 for host in /sys/class/scsi_host/*; do echo "- - -" | sudo tee $host/scan; ls /dev/sd* ; done
 lsblk

Add Option to scsi_id.config

 echo options=-g>>/etc/scsi_id.config

This ensures the SCSI devices are configured as trusted.

Get Device ID

 /usr/lib/udev/scsi_id -g -u -d /dev/sdb
 36000c29654b6aa069e1553a3251c4f56
  • Do this for each disk (sdb,sdc,sdd...).

Create 99-oracle-asmdevices.rules

Create\Edit: /etc/udev/rules.d/99-oracle-asmdevices.rules

For each disk:

  • Set RESULT to the Device ID string from above.
  • Set\increment the SYMLINK name for each disk.
 KERNEL=="sd*1",
 SUBSYSTEM=="block",
 PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",
 RESULT=="36000c29654b6aa069e1553a3251c4f56",
 SYMLINK+="asm-disk01",
 OWNER="grid",
 GROUP="asmadmin",
 MODE="0660"

The above goes on one line.

3 Disk Template

  • KERNEL=="sd*1",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="TBD",SYMLINK+="asm-disk01",OWNER="grid",GROUP="asmadmin",MODE="0660"
  • KERNEL=="sd*1",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="TBD",SYMLINK+="asm-disk02",OWNER="grid",GROUP="asmadmin",MODE="0660"
  • KERNEL=="sd*1",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="TBD",SYMLINK+="asm-disk03",OWNER="grid",GROUP="asmadmin",MODE="0660"

Replace TBD with matching Device ID.

Reload Partition Tables

 /sbin/partprobe /dev/sdb1
 /sbin/partprobe /dev/sdc1
 /sbin/partprobe /dev/sdd1

/sbin/partprobe /dev/sdb1; sleep 1; /sbin/partprobe /dev/sdc1; sleep 1; /sbin/partprobe /dev/sdd1; sleep 1

Test UDev

 udevadm test /block/sdb/sdb1|more
 udevadm test /block/sdc/sdc1|more
 udevadm test /block/sdd/sdd1|more

Restart Udev Service

udevadm control --reload-rules

Check Ownership and Permissions

The symlinks are owned by root, but devices will be owned by grid:asmadmin

  • ls -al /dev/asm*
  • ls -rlt /dev/sd?1

Example Output

[root@lnx01 ~]# ls -al /dev/asm*
lrwxrwxrwx 1 root root 4 Jun 19 12:52 /dev/asm-disk01 -> sdb1
lrwxrwxrwx 1 root root 4 Jun 19 12:54 /dev/asm-disk02 -> sdc1
lrwxrwxrwx 1 root root 4 Jun 19 12:58 /dev/asm-disk03 -> sdd1

[root@lnx01 ~]# ls -rlt /dev/sd?1 
brw-rw---- 1 grid asmadmin 8, 17 Jun 19 12:52 /dev/sdb1
brw-rw---- 1 grid asmadmin 8, 33 Jun 19 12:54 /dev/sdc1
brw-rw---- 1 grid asmadmin 8, 49 Jun 19 12:58 /dev/sdd1

Next Node

Repeat the above procedure for all other RAC node systems. Use the Device ID as displayed from first system.

Reboot

After configuring UDev it is a good idea to reboot all nodes and ensure all the disks look OK via:

  1. Disable CRS autostart: crsctl disable crs
    Important Note: It is wise to disable CRS autostart as ASM can become
    corrupted if it tries to start and the disks are not bound correctly.
  2. Reboot
  3. Test
ls -al /dev/asm*
ls -rlt /dev/sd?1
  1. Start CRS if everything looks OK.

Display Disk Info

Commands to view UDev related disk info.

 blkid
 lsscsi -is
 lshw -class disk
 inxi -Dxx

yum install inxi -y


VMWare UUID Fix

Special thanks to Chris Jolliffe for sharing this!

  1. Power down VM, right click and select edit settings.
  2. Select the options tab and click on General under Advanced.
  3. Click the Configure Parameters Button and verify that the disk.EnableUUID parameter is not present
  4. If present, change Value to “true”, if not present select, the Add Row button and enter the Name disk.EnableUUID and Value true.
  5. Click ok, and ok, and power on the vm.
  6. Now check the scsi_id by /usr/lib/udev/scsi_id -g -u -d /dev/sdb

This method can also be used for VMWare.


Reference

To undo the changes:

  1. Remove /etc/udev/rules.d/10-local.rules and /dev/<YourDiskName>
  2. Run systemctl restart systemd-udevd.

During the GI installation set the ASM_DISKSTRING initialization parameter to /dev/asm-disk* to identify the ASM disks.