Oracle - ASM : Miroir, mon beau miroir, ….

ASM, Cluster RAC Ajouter un commentaire

Oracle - ASM : Miroir, mon beau miroir, ….by albanlepunk

Des fois on pense avoir tout prévu pour sécuriser son infrastructure Oracle : cluster RAC étendu, diskgroups ASM en redondance NORMAL, 3° voting disk sur un partage NFS distinct, ….

Ceinture et bretelle donc …

Sauf que …. dans la vraie vie d’un informaticien (et à plus forte raison d’un consultant Oracle) la loi de Murphy est incontournable.

Une coupure d’une baie de stockage suivie d’une fausse manipulation d’un administrateur plus tard et vous vous retrouvez avec la situation suivante qu’on vous demande à vous de dépatouiller :

SQL> select name, STATE, OFFLINE_DISKS from v$asm_diskgroup;

NAME                           STATE       OFFLINE_DISKS

------------------------------ ----------- -------------

DGDATA                         MOUNTED                 0

DGFRA                          MOUNTED                 0

OCRMIRROR                      MOUNTED                 0

OCRVOTING                      DISMOUNTED              0

Oups, le diskgroup OCRVOTING à l’état DISMOUNTED, ça ne sent pas très bon pour le cluster, d’autant plus  :

- qu’il est impossible de rajouter des disques à ce diskgroup vu qu’il est DISMOUNTED

- qu’il est impossible de le monter, même en mode FORCE, vu qu’il manque des disques

Heureusement l’architecture a été bien pensée, et l’existence d’un groupe de disques OCRMIRROR (contenant pour l’instant l’OCR mirroir comme son nom l’indique) va nous être utile.

On commence par ‘déplacer’ les voting disks sur ce diskgroup

root@dblc00000025vm01 dev]# crsctl replace votedisk +OCRMIRROR

Successful addition of voting disk 86b1b5a6583d4f22bf97f7af26f0ef9d.

Successful deletion of voting disk 2bbecebc98fd4fcbbf96de5cc98bed12.

Successful deletion of voting disk 18c58cd6ce034f3bbfca178420967aa0.

Successfully replaced voting disk group with +OCRMIRROR.

CRS-4266: Voting file(s) successfully replaced

Vérification :

[root@dblc00000025vm01 dev]# crsctl query css votedisk

##  STATE    File Universal Id                File Name Disk group

--  -----    -----------------                --------- ---------

1. ONLINE   86b1b5a6583d4f22bf97f7af26f0ef9d (/dev/oracleasm/disks/OCRVOTING_AB02) [OCRMIRROR]

Located 1 voting disk(s).

On continue en supprimant les entêtes des disques concernés puis en les recréant (attention : vous êtes sensés savoir ce que vous faites quand vous faites ce type de manipulation)

[root@dblc00000025vm01 dev]# dd if=/dev/zero of=/dev/xvdh1 bs=1024k count=10

10+0 records in

10+0 records out

10485760 bytes (10 MB) copied, 0.215507 s, 48.7 MB/s

[root@dblc00000025vm01 dev]# oracleasm querydisk /dev/xvdj1

Device "/dev/xvdj1" is marked an ASM disk with the label "OCRVOTING_AA02"

[root@dblc00000025vm01 dev]# dd if=/dev/zero of=/dev/xvdj1 bs=1024k count=10

10+0 records in

10+0 records out

10485760 bytes (10 MB) copied, 0.199342 s, 52.6 MB/s

[root@dblc00000025vm01 dev]#

[root@dblc00000025vm01 dev]# oracleasm createdisk OCRVOTING_AA01 /dev/xvdh1

Writing disk header: done

Instantiating disk: failed

Clearing disk header: done

[root@dblc00000025vm01 dev]# oracleasm createdisk OCRVOTING_AA02 /dev/xvdj1

Writing disk header: done

Instantiating disk: failed

Clearing disk header: done

[root@dblc00000025vm01 dev]#

[root@dblc00000025vm01 dev]# oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks...

Cleaning disk "OCRVOTING_AA01"

Cleaning disk "OCRVOTING_AA02"

Scanning system for ASM disks...

[root@dblc00000025vm01 dev]# oracleasm listdisks

DATA_AA01

DATA_AB01

FRA_AA01

FRA_AB01

OCRVOTING_AB01

OCRVOTING_AB02

[root@dblc00000025vm01 dev]#

[root@dblc00000025vm01 dev]# oracleasm createdisk OCRVOTING_AA02 /dev/xvdj1

Writing disk header: done

Instantiating disk: done

[root@dblc00000025vm01 dev]# oracleasm createdisk OCRVOTING_AA01 /dev/xvdh1

Writing disk header: done

Instantiating disk: done

[root@dblc00000025vm01 dev]# oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks...

Scanning system for ASM disks...

[root@dblc00000025vm01 dev]# oracleasm listdisks

DATA_AA01

DATA_AB01

FRA_AA01

FRA_AB01

OCRVOTING_AA01

OCRVOTING_AA02

OCRVOTING_AB01

OCRVOTING_AB02

On va créer un diskgroup temporaire pour faire changer le statut des disques concernés dans ASM de ‘MEMBER’ à ‘FORMER’, puis le supprimer.


SQL> create diskgroup DGTEST external redundancy disk '/dev/oracleasm/disks/OCRVOTING_AA01','/dev/oracleasm/disks/OCRVOTING_AA02';
Diskgroup created.

SQL> select name, state from v$asm_diskgroup;

NAME                           STATE

------------------------------ -----------

DGDATA                         MOUNTED

DGFRA                          MOUNTED

OCRMIRROR                      MOUNTED

OCRVOTING                      DISMOUNTED

DGTEST                         MOUNTED

SQL> drop diskgroup DGTEST;

Diskgroup dropped.

SQL> select name,path,header_status from v$asm_disk

NAME                           PATH                                                                                                                                HEADER_STATU

------------------------------ ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------

/dev/oracleasm/disks/OCRVOTING_AA01                                                                                                 FORMER

/dev/oracleasm/disks/OCRVOTING_AA02                                                                                                 FORMER

/dev/oracleasm/disks/OCRVOTING_AB01                                                                                                 MEMBER

/app/voting/vote3                                                                                                                   MEMBER

OCRMIRROR_0000                 /dev/oracleasm/disks/OCRVOTING_AB02                                                                                                 MEMBER

DGFRA_0001                     /dev/oracleasm/disks/FRA_AB01                                                                                                       MEMBER

DGFRA_0002                     /dev/oracleasm/disks/FRA_AA01                                                                                                       MEMBER

DGDATA_0000                    /dev/oracleasm/disks/DATA_AB01                                                                                                      MEMBER

DGDATA_0002                    /dev/oracleasm/disks/DATA_AA01                                                                                                      MEMBER

9 rows selected.

On droppe l’ancien diskgroup OCRVOTING

SQL> drop diskgroup OCRVOTING force including contents;
Diskgroup dropped

On recréé le diskgroup OCRVOTING

SQL> create diskgroup OCRVOTING normal redundancy disk '/dev/oracleasm/disks/OCRVOTING_AA01','/dev/oracleasm/disks/OCRVOTING_AA02';
Diskgroup created.

SQL> select name, state from v$asm_diskgroup;

NAME                           STATE

------------------------------ -----------

DGDATA                         MOUNTED

DGFRA                          MOUNTED

OCRMIRROR                      MOUNTED

OCRVOTING                      MOUNTED

Puis on lui rajoute son disque quorum NFS, il faut modifier le paramètre compatible.asm du diskgroup pour avoir accès à cette fonctionnalité.

SQL> alter diskgroup OCRVOTING SET ATTRIBUTE 'compatible.asm' = '11.2';
Diskgroup altered.

SQL> alter diskgroup OCRVOTING add QUORUM FAILGROUP OCRVOTING_0002 DISK '/app/voting/vote3';

Diskgroup altered.

SQL> select GROUP_NUMBER, NAME, NAME,FAILGROUP from v$asm_disk;

GROUP_NUMBER NAME                           NAME                           FAILGROUP

------------ ------------------------------ ------------------------------ ------------------------------

0

3 OCRMIRROR_0000                 OCRMIRROR_0000                 OCRMIRROR_0000

4 OCRVOTING_0000                 OCRVOTING_0000                 OCRVOTING_0000

4 OCRVOTING_0001                 OCRVOTING_0001                 OCRVOTING_0001

2 DGFRA_0001                     DGFRA_0001                     FG_FRA_AB01

2 DGFRA_0002                     DGFRA_0002                     FG_FRA_AA01

1 DGDATA_0000                    DGDATA_0000                    FG_DATA_AB01

1 DGDATA_0002                    DGDATA_0002                    FG_DATA_AA01

4 OCRVOTING_0002                 OCRVOTING_0002                 OCRVOTING_0002
9 rows selected.

il faut ensuite restaurer le spfile ASM dans le diskgroup, puis

on va redéplacer les voting sur OCRVOTING (Attention seulement en 11.2)

[root@dblc00000025vm01 dev]# crsctl replace votedisk +OCRVOTING
Successful addition of voting disk 4d47939b7e5c4f36bf3bbfcb04ec9c6d.

Successful addition of voting disk 2045390006194f34bf42086af70e5126.

Successful addition of voting disk 311f1a648aa04fa1bf241b204fe5652f.

Successful deletion of voting disk 86b1b5a6583d4f22bf97f7af26f0ef9d.

Successfully replaced voting disk group with +OCRVOTING.

CRS-4266: Voting file(s) successfully replaced

Vérifications

[root@dblc00000025vm01 dev]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group

--  -----    -----------------                --------- ---------

1. ONLINE   4d47939b7e5c4f36bf3bbfcb04ec9c6d (/dev/oracleasm/disks/OCRVOTING_AA01) [OCRVOTING]

2. ONLINE   2045390006194f34bf42086af70e5126 (/dev/oracleasm/disks/OCRVOTING_AA02) [OCRVOTING]

3. ONLINE   311f1a648aa04fa1bf241b204fe5652f (/app/voting/vote3) [OCRVOTING]

Located 3 voting disk(s).

On va supprimmer la référence à l’OCR d’OCRVOTING

[root@dblc00000025vm01 dev]# ocrconfig -delete +OCRVOTING
[root@dblc00000025vm01 dev]# ocrconfig -add +OCRVOTING

[root@dblc00000025vm01 dev]# ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          3

Total space (kbytes)     :     262120

Used space (kbytes)      :       3168

Available space (kbytes) :     258952

ID                       :  603000217

Device/File Name         : +OCRMIRROR

Device/File integrity check succeeded

Device/File Name         : +OCRVOTING

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

[root@dblc00000025vm01 dev]# ocrconfig -delete +OCRMIRROR

[root@dblc00000025vm01 dev]# ocrconfig -add +OCRMIRROR

[root@dblc00000025vm01 dev]# ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          3

Total space (kbytes)     :     262120

Used space (kbytes)      :       3168

Available space (kbytes) :     258952

ID                       :  603000217

Device/File Name         : +OCRVOTING

Device/File integrity check succeeded

Device/File Name         : +OCRMIRROR

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

Un test de reboot des serveurs et c’est bon, c’est reparti !

Faire un commentaire