Troubleshooting ASM disk drop

Written By askMLabs on Wednesday, February 2, 2011 | 4:44 PM

The present article describes how to troubleshoot if you are not able delete an ASM disk from the ASM storage. The troubleshooting technique may vary from person to person and the following article describes the technique i used to drop an ASM disk. There may be so many other possible solutions to the issue, but the one presented here worked for me.

The present issue is that , i have to delete an ASM disk from the ASM storage. The disk is already marked as ASM disk and when i try to unmark it for ASM disk, it throws an error. So i cant delete the OS disk without deleting it from the ASM disks.

Present issue occured in 11gR2 RAC on linux and we are using multi path ASM disks for the storage.

[root@rac01 disks]# /etc/init.d/oracleasm deletedisk DATAVOL1
Removing ASM disk "DATAVOL1": [FAILED]

So i verified the log file /var/log/oracleasm and the content of the log file shows me the following error

Unable to clear disk "DATAVOL1"
Clearing disk header: oracleasm-write-label: Unable to open device "/dev/oracleasm/disks/DATAVOL1": Device or resource busy
failed
Unable to clear disk "DATAVOL1"

I tried creating a diskgroup with this ASM disk and droped it and then tried to delete the ASM disk, but no use.

SQL> create diskgroup test external redundancy disk '/dev/oracleasm/disks/DATAVOL1';

Diskgroup created.

SQL> drop diskgroup test;
Diskgroup dropped.
SQL>

[root@rac01 disks]# /etc/init.d/oracleasm deletedisk DATAVOL1
Removing ASM disk "DATAVOL1": [FAILED]

I verified with the following command if the disk is still marked as ASM disk or not. It is still marked as ASM disk.

[root@rac01 disks]# blkid|grep sd.*oracleasm|while read a b;do echo -n $a$b" scsi_id=";(echo $a|tr -d [:digit:]|tr -d [:]|cut -d"/" -f3|xargs -i scsi_id -g -s /block/{})done | grep -i DATAVOL1;
/dev/sdc:LABEL="DATAVOL1" TYPE="oracleasm" scsi_id=360050768019600fba80000000000036b
/dev/sdf:LABEL="DATAVOL1" TYPE="oracleasm" scsi_id=360050768019600fba80000000000036b
/dev/sdi:LABEL="DATAVOL1" TYPE="oracleasm" scsi_id=360050768019600fba80000000000036b
/dev/sdl:LABEL="DATAVOL1" TYPE="oracleasm" scsi_id=360050768019600fba80000000000036b
/dev/sdo:LABEL="DATAVOL1" TYPE="oracleasm" scsi_id=360050768019600fba80000000000036b
/dev/sdp:LABEL="DATAVOL1" TYPE="oracleasm" scsi_id=360050768019600fba80000000000036b
/dev/sdu:LABEL="DATAVOL1" TYPE="oracleasm" scsi_id=360050768019600fba80000000000036b
/dev/sdx:LABEL="DATAVOL1" TYPE="oracleasm" scsi_id=360050768019600fba80000000000036b
[root@rac01 disks]#

[root@rac01 disks]# multipath -ll

..

..

vpath0 (360050768019600fba80000000000036b) dm-11 IBM,2145
[size=500G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=200][active]
\_ 2:0:7:0 sdo 8:224 [active][ready]
\_ 1:0:7:0 sdp 8:240 [active][ready]
\_ 2:0:6:0 sdu 65:64 [active][ready]
\_ 1:0:6:0 sdx 65:112 [active][ready]
\_ round-robin 0 [prio=40][enabled]
\_ 1:0:4:0 sdc 8:32 [active][ready]
\_ 1:0:5:0 sdf 8:80 [active][ready]
\_ 2:0:4:0 sdi 8:128 [active][ready]
\_ 2:0:5:0 sdl 8:176 [active][ready]
[root@rac01 disks]#

Then i tried clearing the disk with the OS command dd.

[root@rac01 disks]# dd if=/dev/zero of=/dev/dm-11 bs=1024 count=100
100+0 records in
100+0 records out
102400 bytes (102 kB) copied, 0.05526 seconds, 1.9 MB/s
[root@rac01 disks]# /etc/init.d/oracleasm deletedisk DATAVOL1
Removing ASM disk "DATAVOL1": [ OK ]
[root@rac01 disks]# /etc/init.d/oracleasm listdisks
CRSVOL1
DATA1
FRADISK1
FRAVOL1
[root@rac01 disks]#

Now i am able to successfully delete the disk from ASM disk list.

On all other nodes in the RAC , do the following to sync

[root@rac02 disks]# /etc/init.d/oracleasm listdisks
CRSVOL1
DATA1
DATAVOL1
FRADISK1
FRAVOL1
[root@rac02 disks]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]
[root@rac02 disks]# /etc/init.d/oracleasm listdisks
CRSVOL1
DATA1
FRADISK1
FRAVOL1
[root@rac02 disks]#

Content from the log file /var/log/oracleasm now shows ...

Disk "DATAVOL1" defines an unmarked device
Dropping disk: done
Reloading disk partitions: done
Cleaning any stale ASM disks...
Validating disk "CRSVOL1"
Validating disk "DATA1"
Validating disk "FRADISK1"
Validating disk "FRAVOL1"

Hope it helps

SRI

Share this article :

+ comments + 3 comments

Sunil Kumar Nanabala

October 29, 2011 at 1:30 AM

can you explain more about this line

dd if=/dev/zero of=/dev/dm-11 bs=1024 count=100

how about dd command related to that perticular disk ( it didnt show any diskname) i am very curious to know about how its worked

Thanks
Sunil

learnwithme11g

November 3, 2011 at 5:02 PM

Hi Sunil,
If you observe the output of "blkid" command and "multipath" command in the article, the scsi_id is same in both the outputs. This id links to both multipath disks and ASM disknames. In my case the scsi_id is "360050768019600fba80000000000036b".

Thanks
SRI

June 7, 2013 at 3:51 PM

Guys be carefull when you droping the diskgroup,think twice whethare diskgroup is being used or not.

askMLabs