Sometimes back, I noticed that even after shutting down GI (18.104.22.168.x) in a RAC setup, I still can't unmount the filesystem where the GI binaries are located.
Below is the scenario -It looks like the drivers oracleadvm and oracleacfs is using /u01.
After shutting down the GI (22.214.171.124.x) using crsctl stop crs. It's not possible to unmount the /u01 mount point (S/W location of GI).
# umount /u01 umount: /u01: device is busy. (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1)) # cat /etc/oracle-release Oracle Linux Server release 6.6 # uname -r 3.8.13-68.2.2.el6uek.x86_64
-- No output when running lsof/fuser: # lsof /u01 # fuser -mv /u01 # lsof | grep /u01 # lsof | grep u01
-- No loop device, nor any NFS mounts # losetup -a # exportfs -v
-- Nothing is running after "crsctl stop crs" # ps -ef |grep d.bin root 22343 3617 0 11:26 pts/0 00:00:00 grep d.bin # ps -ef |grep oracle root 22345 3617 0 11:26 pts/0 00:00:00 grep oracle
-- Filesystem Mount Options # mount | grep /u01 /dev/mapper/VolGroup-LVDbOra1 on /u01 type ext4 (rw,nodev)
-- Its been observed that following modules are currently loaded into the kernel # lsmod | grep oracle oracleacfs 3308260 0 oracleadvm 508030 0 oracleoks 506741 2 oracleacfs,oracleadvm
File referenced by drivers will not show up on the lsof nor fuser output.
From the MOS Note # Unable to Umount /u01 on Exadata X5 machine (Oracle 126.96.36.199 BP6) (Doc ID 2001832.1) -
After removing the driver modules from the Kernel using modprobe, it is possible to unmount the filesystem:
# modprobe -r oracleadvm # modprobe -r oracleacfs # umount /u01 # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup-LVDbSys1 12G 1.4G 9.8G 12% / tmpfs 1.8G 0 1.8G 0% /dev/shm /dev/sda1 477M 40M 408M 9% /boot
BUT, it turns out that the recommended method is to -
- First stop the Oracle stack: crsctl stop crs
- Then unload the drivers: acfsload stop
- Lastly unmount the native file systems (e.g., "/u01"): umount /u01