반응형
사내 테스트 환경을 정리 한답시고 디렉토리들을 정리 하다가, 모르고 사용중인 container를 삭제 하였다.

워낙에 여러 사람들이 테스트로 사용하는 환경이라... 그 밑에 container가 존재할지는 몰랐다.

그렇게 정리를 하고 있었는데 어느 순간 V9.1 서버가 접근이 안되고 diag가 엄청나게 발생하고 있다고 해서 확인해 봤더니 ..

container가 없어서 DB가 Down이 되었으며, instance는 기동이 되나 DB는 기동이 되질 않는 상황이었다.

diag를 확인해 보니, 다행히 해당 테이블 유저가 생성시킨 테스트용 테이블 스페이스였다. 복구는 필요치 않았으며, 뭐 복구를 할려고 해도 백업이 없으니 복구는 힘들고...

찾아 보니 restart 라는 명령어 뒤에 drop pending tablesapces 라는 옵션이 있는 것을 확인 할 수 있었다.


[db2ins91|db2ins91]/home/db2ins91/sqllib/db2dump>db2 "restart database sample drop pending tablespaces ( TS_MXG_8K_1) "
DB20000I  The RESTART DATABASE command completed successfully.
[db2ins91|db2ins91]/home/db2ins91/sqllib/db2dump>db2 connect to sample

 Database Connection Information

 Database server        = DB2/AIX64 9.1.4
 SQL authorization ID   = DB2INS91
 Local database alias   = SAMPLE
  

db2diag.log 파일 내용의 일부


2011-05-10-20.48.19.090712+540 I359505281A485     LEVEL: Error
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbDMSDoContainerOp, probe:810
DATA #1 : String, 115 bytes
Error checking container 0 (/home/db2ins91/DB2TEST/REPOSITORY/URDBscipt/TS_MXG_8K_1.DMS) for tbsp 4.  Rc = 870F0011
2011-05-10-20.48.19.091152+540 I359505767A441     LEVEL: Error
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbDMSStartPool, probe:800
MESSAGE : ZRC=0x8402001E=-2080243682=SQLB_CONTAINER_NOT_ACCESSIBLE
          "Container not accessible"
2011-05-10-20.48.19.091593+540 I359506209A391     LEVEL: Error
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbDMSStartPool, probe:800
DATA #1 : String, 27 bytes
Tablespace 4 (TS_MXG_8K_1)
2011-05-10-20.48.19.091925+540 E359506601A517     LEVEL: Error
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbStartPools, probe:30
MESSAGE : ADM6023I  The table space "TS_MXG_8K_1" (ID "4") is in state 0x"0". 
          The table space cannot be accessed.  Refer to the documentation for
          SQLCODE -290.
2011-05-10-20.48.19.094468+540 I359507119A438     LEVEL: Error
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbStartPools, probe:30
MESSAGE : ZRC=0x8402001E=-2080243682=SQLB_CONTAINER_NOT_ACCESSIBLE
          "Container not accessible"
2011-05-10-20.48.19.094863+540 E359507558A531     LEVEL: Severe
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbStartPoolsErrorHandling, probe:55
MESSAGE : ADM6023I  The table space "TS_MXG_8K_1" (ID "4") is in state 0x"0". 
          The table space cannot be accessed.  Refer to the documentation for
          SQLCODE -290.
2011-05-10-20.48.19.114153+540 E359508090A1277    LEVEL: Error
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbStartPools, probe:63
MESSAGE : ADM6049E  The database cannot be restarted because one or more table
          spaces cannot be brought online. To restart the database specify the
          "DROP PENDING TABLESPACES" option on the RESTART DATABASE command. 
          WARNING! PUTTING A TABLESPACE INTO THE DROP PENDING STATE MEANS THAT
          NO FURTHER ACCESS TO THE TABLESPACE WILL BE ALLOWED. ITS CONTENTS
          WILL BE INACCESSIBLE THROUGHOUT THE REMAINDER OF THE LIFE OF THE
          TABLESPACE AND THE ONLY OPERATION THAT WILL BE ALLOWED ON IT IS "DROP
          TABLESPACE".  There is no way in which it can be brought back.  It is
          important that you consider the consequences of this action as data
          can be lost as a result. Before proceeding consult the DB2
          documentation and contact IBM support if necessary.  The table spaces
          to specify in the DROP PENDING TABLESPACES list are: "TS_MXG_8K_1 ".
2011-05-10-20.48.19.114872+540 I359509368A454     LEVEL: Error
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbStartPools, probe:182
MESSAGE : ZRC=0x80020035=-2147352523=SQLB_NOT_ALLOWED_RECOVERY_P
          "Access not allowed. Tblspc Recovery Pend."
2011-05-10-20.48.19.115263+540 I359509823A448     LEVEL: Error
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, buffer pool services, sqlbinit, probe:590
MESSAGE : ZRC=0x80020035=-2147352523=SQLB_NOT_ALLOWED_RECOVERY_P
          "Access not allowed. Tblspc Recovery Pend."
2011-05-10-20.48.19.277163+540 I359510272A424     LEVEL: Severe
PID     : 1933344              TID  : 1           PROC : db2agent (SAMPLE) 0
INSTANCE: db2ins91             NODE : 000
APPHDL  : 0-295                APPID: *LOCAL.db2ins91.110510114818
AUTHID  : DB2INS91
FUNCTION: DB2 UDB, base sys utilities, sqledint, probe:160
DATA #1 : Hexdump, 4 bytes
0x0FFFFFFFFFFEA7B0 : 8002 0035                                  ...5

오랜만에 DB 문제가 발생한걸 처리 하니 기분이 묘하네요...
정말 백업/복구 오랜만입니다. ㅎㅎ

DB가 장애 났을때의 그 오싹함과, 등줄기의 소름, 복구시의 그 쾌감은 없지만 그래도 오랜만에 DB 이상 문제에 당면해서 조치를 하니 재미는 있네요.

역시 O/S상의 rm 명령어는 무서운 명령어 입니다. ㅎㅎ
반응형

+ Recent posts