SpecialTopics

UDev Configuration

Overview

UDev guarantees persistent device names across reboots regardless of their order of discovery. This is vital for ASM storage devices. UDev also ensures the devices have the correct ownership for ASM. This page covers configuring Oracle Linux UDev to manage disks for ASM.

Linux 8 Procedure
Linux 7 Procedure
Display Device Names (/dev/sd?) and UUIDs

Oracle Reference: Doc ID 2721585.1

UDev Fun Facts!

Oracle fully supports the Linux 7\8 methods detailed here using the UDev SYMLINK method as opposed to the Linux 5\6.x legacy method of using the mapper architecture (per 3-17731438021).
ASMLib has been deprecated in favor of UDev or ASMFD.
Disks for ASM only need to be partitioned, i.e. not formatted and mounted.
Udev is not required for standard disks. To configure\mount a standard disk using its UUID go here.
If you are using VMWare you may need to do this to display the UUID.

Linux 8 Procedure

The scenario below shows implementing Udev with 3 shared disks.

Confirm Hardware Detected

 /sbin/partprobe
 for host in /sys/class/scsi_host/*; do echo "- - -" | sudo tee $host/scan; ls /dev/sd* ; done
 lsblk

Add Option to scsi_id.config

 echo options=-g>>/etc/scsi_id.config

This ensures the SCSI devices are configured as trusted.

Get Device ID

 udevadm info --query=all --name=/dev/sdb | egrep "WWN|SERIAL"
 Example: ID_SERIAL=36000c29654b6aa069e1553a3251c4f56

Note the value of ID_SERIAL.
Do this for each disk (sdb,sdc,sdd...).

Create 96-asm.rules

Create\Edit: /etc/udev/rules.d/96-asm.rules
You may wish to consider ansible to maintain files across multiple Linux systems.

For each disk:

Set ENV{ID_SERIAL} to the ID_SERIAL string from above.
Set\increment the SYMLINK name for each disk.

 KERNEL=="sd*1",
 SUBSYSTEM=="block",
 ENV{ID_SERIAL}=="36000c29654b6aa069e1553a3251c4f56",
 SYMLINK+="asm-disk01",
 OWNER="grid",
 GROUP="asmadmin",
 MODE="0660",
 OPTIONS:="nowatch"

The above goes on one line.

3 Disk Template

 KERNEL=="sd*",SUBSYSTEM=="block",ENV{ID_SERIAL}=="X",SYMLINK+="asm-disk01",OWNER="grid",GROUP="asmadmin",MODE="0660",OPTIONS:="nowatch"
 KERNEL=="sd*",SUBSYSTEM=="block",ENV{ID_SERIAL}=="X",SYMLINK+="asm-disk02",OWNER="grid",GROUP="asmadmin",MODE="0660",OPTIONS:="nowatch"
 KERNEL=="sd*",SUBSYSTEM=="block",ENV{ID_SERIAL}=="X",SYMLINK+="asm-disk03",OWNER="grid",GROUP="asmadmin",MODE="0660",OPTIONS:="nowatch"

Replace X with matching ID_SERIAL value.

UDev Check

Ensure udev shows bound to correct disk and UUID.

 udevadm test /block/sdb/sdb1|more
 ...
 DEVNAME=/dev/sdb1
 ID_SERIAL=36000c29654b6aa069e1553a3251c4f56

Do this for each disk (sdb,sdc,sdd...).

Reload the udev/rules

 udevadm control --reload-rules
 udevadm trigger --type=devices --action=change

Test

Ensure asm-disknn bound to correct Linux volume.

 ls -al /dev/asm*
 lrwxrwxrwx 1 root root 4 Oct 14 13:05 /dev/asm-disk01 -> sdb1
 lrwxrwxrwx 1 root root 4 Oct 14 13:05 /dev/asm-disk02 -> sdc1
 lrwxrwxrwx 1 root root 4 Oct 14 13:05 /dev/asm-disk03 -> sdd1

Ensure devices devices owned by grid:asmadmin.

 ls -rlt /dev/sd?1
 brw-rw---- 1 grid asmadmin 8, 49 Oct 14 13:05 /dev/sdd1
 brw-rw---- 1 grid asmadmin 8, 17 Oct 14 13:05 /dev/sdb1
 brw-rw---- 1 grid asmadmin 8, 33 Oct 14 13:05 /dev/sdc1

They may not appear in sequential order. That is OK.

Display Disk Names (/dev/sd?) and UUIDs

 lsscsi -iss
 [0:0:0:0]    ...   /dev/sda   36000c2992320fd0bc87e8942736ecf45   133GB
 [3:0:0:0]    ...   /dev/sdb   36000c29fba0a99ebe6975d8851d8d7f6  10.7GB
 [4:0:0:0]    ...   /dev/sdc   36000c29c72b5c385814445db0d5bbf36  11.8GB
 [4:0:1:0]    ...   /dev/sdd   36000c2971d606bc7ab3c56496a8d838d  12.8GB
 [4:0:2:0]    ...   /dev/sde   36000c2905d53274177de130279e2a615  13.9GB

 lsblk -f
 NAME        FSTYPE      LABEL UUID                                   MOUNTPOINT
 sda
 ├─sda1      xfs               d95000ba-2315-45f7-a974-ec47612046dd   /boot
 └─sda2      LVM2_member       dVRyC6-Bvny-e0eY-dMhn-zcMI-ebtE-RABySR
   ├─ol-root xfs               9523ea76-5f79-4a48-8da8-219c2ab27f3e   /
   ├─ol-swap swap              b2cca244-6255-4c9e-9159-b50ad31a1a29   [SWAP]
   └─ol-home xfs               762149de-4627-43ea-bc89-3d0d65e5ca11   /home
 sdb
 └─sdb1      ext4              a3764216-babd-451a-88c5-a66eb7abaa64   /u02

Next Node

Repeat the above procedure for all other RAC node systems. Use the Device ID as displayed from first system.

Reboot

After configuring UDev it is a good idea to reboot all nodes and ensure all the disks look OK via:

Disable CRS autostart: crsctl disable crs
Important Note: It is wise to disable CRS autostart as ASM can become
corrupted if it tries to start and the disks are not bound correctly.
Reboot
Test (as shown above)

ls -al /dev/asm*
ls -rlt /dev/sd?1

Start CRS if everything looks OK.

Linux 7 Procedure

The scenario below shows implementing Udev with 3 shared disks.

Confirm Hardware Detected

 /sbin/partprobe
 for host in /sys/class/scsi_host/*; do echo "- - -" | sudo tee $host/scan; ls /dev/sd* ; done
 lsblk

Add Option to scsi_id.config

 echo options=-g>>/etc/scsi_id.config

This ensures the SCSI devices are configured as trusted.

Get Device ID

 /usr/lib/udev/scsi_id -g -u -d /dev/sdb
 36000c29654b6aa069e1553a3251c4f56

Do this for each disk (sdb,sdc,sdd...).

Create 99-oracle-asmdevices.rules

Create\Edit: /etc/udev/rules.d/99-oracle-asmdevices.rules

For each disk:

Set RESULT to the Device ID string from above.
Set\increment the SYMLINK name for each disk.

 KERNEL=="sd*1",
 SUBSYSTEM=="block",
 PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",
 RESULT=="36000c29654b6aa069e1553a3251c4f56",
 SYMLINK+="asm-disk01",
 OWNER="grid",
 GROUP="asmadmin",
 MODE="0660"

The above goes on one line.

3 Disk Template

KERNEL=="sd*1",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="TBD",SYMLINK+="asm-disk01",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*1",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="TBD",SYMLINK+="asm-disk02",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*1",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="TBD",SYMLINK+="asm-disk03",OWNER="grid",GROUP="asmadmin",MODE="0660"

Replace TBD with matching Device ID.

Reload Partition Tables

 /sbin/partprobe /dev/sdb1
 /sbin/partprobe /dev/sdc1
 /sbin/partprobe /dev/sdd1

/sbin/partprobe /dev/sdb1; sleep 1; /sbin/partprobe /dev/sdc1; sleep 1; /sbin/partprobe /dev/sdd1; sleep 1

Test UDev

 udevadm test /block/sdb/sdb1|more
 udevadm test /block/sdc/sdc1|more
 udevadm test /block/sdd/sdd1|more

Restart Udev Service

udevadm control --reload-rules

Check Ownership and Permissions

The symlinks are owned by root, but devices will be owned by grid:asmadmin

ls -al /dev/asm*
ls -rlt /dev/sd?1

Example Output

[root@lnx01 ~]# ls -al /dev/asm*
lrwxrwxrwx 1 root root 4 Jun 19 12:52 /dev/asm-disk01 -> sdb1
lrwxrwxrwx 1 root root 4 Jun 19 12:54 /dev/asm-disk02 -> sdc1
lrwxrwxrwx 1 root root 4 Jun 19 12:58 /dev/asm-disk03 -> sdd1

[root@lnx01 ~]# ls -rlt /dev/sd?1 
brw-rw---- 1 grid asmadmin 8, 17 Jun 19 12:52 /dev/sdb1
brw-rw---- 1 grid asmadmin 8, 33 Jun 19 12:54 /dev/sdc1
brw-rw---- 1 grid asmadmin 8, 49 Jun 19 12:58 /dev/sdd1

Next Node

Repeat the above procedure for all other RAC node systems. Use the Device ID as displayed from first system.

Reboot

After configuring UDev it is a good idea to reboot all nodes and ensure all the disks look OK via:

Disable CRS autostart: crsctl disable crs
Important Note: It is wise to disable CRS autostart as ASM can become
corrupted if it tries to start and the disks are not bound correctly.
Reboot
Test

ls -al /dev/asm*
ls -rlt /dev/sd?1

Start CRS if everything looks OK.

Display Disk Info

Commands to view UDev related disk info.

 blkid
 lsscsi -is
 lshw -class disk
 inxi -Dxx

yum install inxi -y

VMWare UUID Fix

VMWare 6.7

Power down VM. Right click and select "Edit Settings".
Select the "VM Options" tab and expand the "Advanced" section.
Under "Configuration Parameters" click on "Edit Configuration".
Verify that the "disk.EnableUUID" parameter is not present. If it is present, make sure it is set to TRUE and skip the next step.
Click "Add Configuration Params" and add the name "disk.EnableUUID" and the value "TRUE".
Click on OK and then OK again.
Power on the VM.
Verify the scsi_id using command "/usr/lib/udev/scsi_id -g -u -d /dev/sdb".

VMWare (earlier versions)

Special thanks to Chris Jolliffe for sharing this!

Power down VM, right click and select edit settings.
Select the options tab and click on General under Advanced.
Click the Configure Parameters Button and verify that the disk.EnableUUID parameter is not present
If present, change Value to “true”, if not present select, the Add Row button and enter the Name disk.EnableUUID and Value true.
Click ok, and ok, and power on the vm.
Now check the scsi_id by /usr/lib/udev/scsi_id -g -u -d /dev/sdb

This method can also be used for VMWare.

Reference

For Linux 8 see: Doc ID 2721585.1.
Linux Disk Discovery 101
UDev MOS Doc IDs: 2056427.1, 1528148.1.
UDev Links:
Create ASM\Multipath Devices | Udev Device Manager | oracle-base.com |
Install Guide for Linux | wiki.archlinux.org.

To undo the changes:

Remove /etc/udev/rules.d/10-local.rules and /dev/<YourDiskName>
Run systemctl restart systemd-udevd.

During the GI installation set the ASM_DISKSTRING initialization parameter to /dev/asm-disk* to identify the ASM disks.