This page discuesses an issue with multipathing in Oracle 11.2.0.2 on Red Hat Enterprise Linux 5 Update 6
We had configured /etc/multipath.conf to set the UID and GID for multipath devices provisioned to support ASM disk groups.
[oracle@server1]$ ls -l /dev/mapper/ora_data1 brw-rw---- 1 oracle dba 253, 3 Jun 14 09:00 ora_data1
I decided that I wanted to change the GID from the dba group to the oinstall group, so I updated the entry in /etc/multipath.conf.from
multipath {
wwid 360060d40053211000000476200000a09
alias ora_data1
uid 500
gid 201
}
to
multipath {
wwid 360060d40053211000000476200000a09
alias ora_data1
uid 500
gid 202
}
The server was subsequently rebooted, but the device group was still dba:
[oracle@server1]$ ls -l /dev/mapper/ora_data1 brw-rw---- 1 oracle dba 253, 3 Jun 14 09:00 ora_data1
The most obvious possibility was that the group was being overwritten in /etc/rc.local, but that file only contained the default entry:
touch /var/lock/subsys/local
Suspicious of the system administrators I searched through udev on the off-chance they had figured out how to change ownerships in there - but none of the files had changed since the operating system was installed.
Finally we discovered that the server was configured to boot from SAN. This means that some of the files required for the initial boot process must be copied into the boot partition. Updating them in /etc has no effect as the kernel has to read the files during the boot phase at which time /etc is unavailable.
Having discovered that /etc/multipath.conf was being ignored, my next concern was whether this affected any other configuration files.
Apparently this issue affects RHEL5U5 and above. To check which files are affected first determine the current kernel using uname -r. For example:
[root@server1]# uname -r 2.6.18-238.1.1.el5
Identify the current kernel image in the /boot directory. In this example the kernel image was:
/boot/initrd-2.6.18-238.1.1.el5.img
Change to another directory e.g. /tmp/initrd. This step is important to avoid inadvertently overwriting the /boot directory
[root@server1]# mkdir /tmp/initrd [root@server1]# cd /tmp/initrdThe following command extracts the contents of the kernel image
zcat /boot/initrd-2.6.18-238.1.1.el5.img | cpio -id
This command will create a set of directories under /tmp/initrd and extract the configuration files into them. In our example there were four files in this directory:
It is also worth noting that both /etc/multipath.conf and /var/lib/multipath/bindings file are copied to the boot partition.
If you update /etc/multipath.conf it will be necessary to rebuild the initrd so that the changes are applied the next time the node is rebooted.
Before rebuilding the initrd, ensure you have backed up the previous image. For example:
cp /boot/initrd-2.6.18-238.1.1.el5.img /boot/initrd-2.6.18-238.1.1.el5.img.bak
Rebuild the initrd using the following command:
mkinitrd /boot/initrd-$(uname -r).img $(uname -r)
So far this is the only boot from SAN configuration I have encountered so it is too early to say if these are the only configuration files that can be overridden in the boot partition. But at least I now know how to check