반응형
사내에서 테스트를 위해서 AIX 9.7에 테이블 스페이스를 생성 중 DIsk Full 에러 메세지를 만나게 되었습니다.

디스크 공간도 넉넉하였고, 유저 Limit도 걸려 있지 않았는데, 에러가 발생하면서 테이블 스페이스가 만들어 지지 않음.

db2ins97:/home/db2ins97>ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        4194304
memory(kbytes)       unlimited
coredump(blocks)     unlimited
nofiles(descriptors) unlimited


db2ins97:/home/db2ins97/DB2INS97>df -k                  
Filesystem    1024-blocks      Free %Used    Iused %Iused Mounted on
/dev/hd4          2097152   1880984   11%     5842     1% /
/dev/hd2          1966080    109436   95%    43705     9% /usr
/dev/hd9var       1048576    791436   25%     6563     3% /var
/dev/hd3          1048576    990664    6%       78     1% /tmp
/dev/hd1         60817408  31785784   48%    30680     1% /home
/proc                   -         -    -         -     -  /proc
/dev/hd10opt      1966080    141028   93%     9440     2% /opt

db2ins97:/home/db2ins97/DB2INS97>db2 -tvf CRE_TBS.DB2
CREATE TABLESPACE TS_MXG_8K PAGESIZE 8192 MANAGED BY DATABASE USING (FILE '/home/db2ins97/db2ins97/TBS/MXG_8K.DMS' 1310720) EXTENTSIZE 32 PREFETCHSIZE AUTOMATIC BUFFERPOOL BPD8K NO FILE SYSTEM CACHING
DB21034E  The command was processed as an SQL statement because it was not a
valid Command Line Processor command.  During SQL processing it returned:
SQL0968C  The file system is full.  SQLSTATE=57011

while 문장을 이용해서 파일의 사이즈를 관찰해 본 결과 2G 정도 파일이 생성되다가 에러가 발생함.

total 4120330
drwxr-xr-x    2 db2ins97 db2adm          512 Mar 22 13:37 .
drwxrwxr-x    4 db2ins97 db2adm          512 Mar 22 13:17 ..
-rw-------    1 db2ins97 db2adm   2109604864 Mar 22 13:41 MXG_8K.DMS
total 4172042
drwxr-xr-x    2 db2ins97 db2adm          512 Mar 22 13:37 .
drwxrwxr-x    4 db2ins97 db2adm          512 Mar 22 13:17 ..
-rw-------    1 db2ins97 db2adm   2136081408 Mar 22 13:42 MXG_8K.DMS
ls: 0653-341 The file ./MXG_8K.DMS does not exist.
total 2
drwxr-xr-x    2 db2ins97 db2adm          512 Mar 22 13:37 .
drwxrwxr-x    4 db2ins97 db2adm          512 Mar 22 13:17 ..
total 2
drwxr-xr-x    2 db2ins97 db2adm          512 Mar 22 13:42 .
drwxrwxr-x    4 db2ins97 db2adm          512 Mar 22 13:17 ..

해당 시점의 db2diag.log 파일의 내용을 살펴 보면

2011-03-22-13.53.31.971346+540 E339134555A1219    LEVEL: Error (OS)
PID     : 778356               TID  : 2829        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : MFD
APPHDL  : 0-7                  APPID: *LOCAL.db2ins97.110322045134
AUTHID  : DB2INS97
EDUID   : 2829                 EDUNAME: db2agent (MFD) 0
FUNCTION: DB2 UDB, oper system services, sqlowrite, probe:70
MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
          DIA8312C Disk was full.
CALLED  : OS, -, write
OSERR   : ENOSPC (28) "No space left on device"
DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
  File Handle              = 90
  File System Block Size   = 512 bytes
  File System Type         = jfs
  File Handle Flags :
    Require Sector Align   = No
    DIO/CIO Mode           = No
    Raw Block Device       = No
    Reserved Handle        = No
    Flush On Close         = Yes
    Thread-Level Lock      = No
    Write-through Mode     = No
    File Not Tracked       = No
DATA #2 : unsigned integer, 8 bytes
131072
DATA #3 : signed integer, 8 bytes
130047
DATA #4 : String, 105 bytes
Search for ossError*Analysis probe point after this log entry for further
self-diagnosis of this problem.
2011-03-22-13.53.32.112180+540 I339135775A2681    LEVEL: Error (OS)
PID     : 778356               TID  : 2829        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : MFD
APPHDL  : 0-7                  APPID: *LOCAL.db2ins97.110322045134
AUTHID  : DB2INS97
EDUID   : 2829                 EDUNAME: db2agent (MFD) 0
FUNCTION: DB2 Common, OSSe, ossErrorIOAnalysis, probe:100
CALLED  : OS, -, write
OSERR   : ENOSPC (28) "No space left on device"
DATA #1 : String, 146 bytes
A total of 5 analysis will be performed :
 - User info
 - ulimit info
 - Target file info
 - I/O attempt
 - File system
 Target file handle = 90
DATA #2 : String, 184 bytes
  Real user ID of current process       = 205
  Effective user ID of current process  = 205
  Real group ID of current process      = 202
  Effective group ID of current process = 202
DATA #3 : String, 383 bytes
Current process limits (unit in bytes except for nofiles) :
  mem     (S/H) = unlimited / unlimited
  core    (S/H) = 8589934590 / 8589934590
  cpu     (S/H) = unlimited / unlimited
  data    (S/H) = unlimited / unlimited
  fsize   (S/H) = unlimited / unlimited
  nofiles (S/H) = unlimited / unlimited
  stack   (S/H) = 4294967296 / 4294967296
  rss     (S/H) = unlimited / unlimited

DATA #4 : String, 263 bytes
Target File Information :
  Size               = 2147483647
  Link               = No
  Reference path     = N/A
  Type               = 0x8000
  Permissions        = rw-------
  UID                = 205
  GID                = 202
  Last modified time = 1300769611
DATA #5 : String, 33 bytes
I/O attempt not implemented yet.
DATA #6 : String, 375 bytes
File System Information of the target file :
  Block size        = 512 bytes
  Total size        = 62277025792 bytes
  Free size         = 30389096448 bytes
  Total # of inodes = 15204352
  FS name           = N/A
  Mount point       = /home
  FSID (major,minor)= 10, 8
  FS type name      = jfs
  DIO/CIO mount opt = None
  Device type       = N/A
  FS type           = 0x5
CALLSTCK:
  [0] 0x090000000564AEFC pdOSSeLoggingCallback + 0x34
  [1] 0x0900000001F86424 oss_log__FP9OSSLogFacUiN32UlN26iPPc + 0x1C4
  [2] 0x0900000001F86900 ossLogSysRC + 0xA0
  [3] 0x0900000001FA608C ossErrorIOAnalysis__FCPC21OSSErrorAnalysisParam + 0xC4C
  [4] 0x0900000001FA8108 ossErrorAnalysis@AF12_1 + 0x48
  [5] 0x090000000743B52C sqloSystemErrorHandler + 0x54
  [6] 0x0900000007F325C0 sqloSystemErrorHandler@glue7A9 + 0x124
  [7] 0x09000000056CA354 sqlowrite + 0x158
  [8] 0x0900000006A075D0 @79@sqloFilePreallocationUsingSlowWrite__FCP12SQLO_FHANDLECPLCLb + 0x198
  [9] 0x0900000006A05DB8 @79@sqlo_file_ext_and_commit__FCP12SQLO_FHANDLECLT2 + 0x148
2011-03-22-13.53.32.113675+540 I339138457A1120    LEVEL: Error
PID     : 778356               TID  : 2829        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : MFD
APPHDL  : 0-7                  APPID: *LOCAL.db2ins97.110322045134
AUTHID  : DB2INS97
EDUID   : 2829                 EDUNAME: db2agent (MFD) 0
FUNCTION: DB2 UDB, oper system services, sqloFilePreallocationUsingSlowWrite, probe:100
MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
          DIA8312C Disk was full.
DATA #1 : Codepath, 8 bytes
1:5:25
DATA #2 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
  File Handle              = 90
  File System Block Size   = 512 bytes
  File System Type         = jfs
  File Handle Flags :
    Require Sector Align   = No
    DIO/CIO Mode           = No
    Raw Block Device       = No
    Reserved Handle        = No
    Flush On Close         = Yes
    Thread-Level Lock      = No
    Write-through Mode     = No
    File Not Tracked       = No
DATA #3 : unsigned integer, 8 bytes
10737417216
DATA #4 : unsigned integer, 8 bytes
8590064640
DATA #5 : unsigned integer, 4 bytes
0
2011-03-22-13.53.32.114786+540 I339139578A541     LEVEL: Error
PID     : 778356               TID  : 2829        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : MFD
APPHDL  : 0-7                  APPID: *LOCAL.db2ins97.110322045134
AUTHID  : DB2INS97
EDUID   : 2829                 EDUNAME: db2agent (MFD) 0
FUNCTION: DB2 UDB, oper system services, sqlo_file_ext_and_commit, probe:100
MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
          DIA8312C Disk was full.
DATA #1 : Codepath, 8 bytes
5:25:26
2011-03-22-13.53.45.153259+540 I339140120A498     LEVEL: Error
PID     : 778356               TID  : 2829        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : MFD
APPHDL  : 0-7                  APPID: *LOCAL.db2ins97.110322045134
AUTHID  : DB2INS97
EDUID   : 2829                 EDUNAME: db2agent (MFD) 0
FUNCTION: DB2 UDB, buffer pool services, sqlbDMSCreatePool, probe:800
MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
          DIA8312C Disk was full.
2011-03-22-13.53.45.155125+540 I339140619A438     LEVEL: Error
PID     : 778356               TID  : 2829        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : MFD
APPHDL  : 0-7                  APPID: *LOCAL.db2ins97.110322045134
AUTHID  : DB2INS97
EDUID   : 2829                 EDUNAME: db2agent (MFD) 0
FUNCTION: DB2 UDB, buffer pool services, sqlbDMSCreatePool, probe:800
DATA #1 : String, 7 bytes
 Page=4
2011-03-22-13.53.45.155640+540 I339141058A495     LEVEL: Error
PID     : 778356               TID  : 2829        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : MFD
APPHDL  : 0-7                  APPID: *LOCAL.db2ins97.110322045134
AUTHID  : DB2INS97
EDUID   : 2829                 EDUNAME: db2agent (MFD) 0
FUNCTION: DB2 UDB, buffer pool services, sqlbCreatePool, probe:950
MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
          DIA8312C Disk was full.
2011-03-22-13.53.45.237567+540 I339141554A491     LEVEL: Severe
PID     : 778356               TID  : 2829        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : MFD
APPHDL  : 0-7                  APPID: *LOCAL.db2ins97.110322045134
AUTHID  : DB2INS97
EDUID   : 2829                 EDUNAME: db2agent (MFD) 0
FUNCTION: DB2 UDB, data management, sqldPoolCreate, probe:273
RETCODE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
          DIA8312C Disk was full.
2011-03-22-13.57.33.714235+540 I339142046A416     LEVEL: Error
PID     : 905218               TID  : 1           PROC : db2bp
INSTANCE: db2ins97             NODE : 000
APPID   : *LOCAL.db2ins97.110322033215
EDUID   : 1
FUNCTION: DB2 UDB, base sys utilities, sqleriar_database, probe:9
RETCODE : ZRC=0x8005006D=-2147155859=SQLE_CA_BUILT
          "SQLCA has been built and saved in component specific control block."
2011-03-22-13.58.09.744630+540 I339142463A416     LEVEL: Error
PID     : 745476               TID  : 1           PROC : db2bp
INSTANCE: db2ins97             NODE : 000
APPID   : *LOCAL.db2ins97.110322043154
EDUID   : 1
FUNCTION: DB2 UDB, base sys utilities, sqleriar_database, probe:9
RETCODE : ZRC=0x8005006D=-2147155859=SQLE_CA_BUILT
          "SQLCA has been built and saved in component specific control block."
2011-03-22-13.58.15.828149+540 I339142880A481     LEVEL: Event
PID     : 778356               TID  : 6942        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000
APPHDL  : 0-27                 APPID: *LOCAL.db2ins97.110322045815
AUTHID  : DB2INS97
EDUID   : 6942                 EDUNAME: db2agent (instance) 0
FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:20
CHANGE  : CFG DB SAMPLE  : "Database_memory" From: "20" <automatic>  To: "35904" <automatic>
2011-03-22-13.58.16.844242+540 E339143362A447     LEVEL: Event
PID     : 778356               TID  : 6942        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000
APPHDL  : 0-27                 APPID: *LOCAL.db2ins97.110322045815
AUTHID  : DB2INS97
EDUID   : 6942                 EDUNAME: db2agent (instance) 0
FUNCTION: DB2 UDB, base sys utilities, sqeLocalDatabase::FirstConnect, probe:1000
START   : DATABASE: SAMPLE   : ACTIVATED: NO
2011-03-22-13.59.08.007800+540 E339143810A446     LEVEL: Event
PID     : 778356               TID  : 6942        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000
APPHDL  : 0-27                 APPID: *LOCAL.db2ins97.110322045815
AUTHID  : DB2INS97
EDUID   : 6942                 EDUNAME: db2agent (idle) 0
FUNCTION: DB2 UDB, base sys utilities, sqeLocalDatabase::TermDbConnect, probe:2000
STOP    : DATABASE: SAMPLE   : DEACTIVATED: NO
2011-03-22-13.59.09.215865+540 E339144257A467     LEVEL: Event
PID     : 778356               TID  : 7206        PROC : db2sysc 0
INSTANCE: db2ins97             NODE : 000         DB   : SAMPLE
APPHDL  : 0-28                 APPID: *LOCAL.db2ins97.110322045908
AUTHID  : DB2INS97
EDUID   : 7206                 EDUNAME: db2agent (SAMPLE) 0
FUNCTION: DB2 UDB, base sys utilities, sqeLocalDatabase::FirstConnect, probe:1000
START   : DATABASE: SAMPLE   : ACTIVATED: NO


위와 같은 에러 메세지가 발생 함.

/home의 Total Size는 60G 이고 30G의 여유 공간이 있는데 2G 까지 파일을 만들고 그 이후 Disk Full 메세지를 출력하는 것이다.

구글링을 하다 보니 결과는 약간 틀리나... jfs일 경우 파일의 제약이 있다는 문서를 만나 볼수 있었다.

파일 시스템을 변경할 수는 없으므로, 1G짜리 10개로 10G 짜리 테이블 스페이스를 생성해서 테스트를 진행하게 됨.

참고 웹 문서 :  http://dbaspot.com/forums/db2-udb/168492-problem-db2-8-2-aix-file-system-full-sql0968c.html


파일 시스템을 확인해 보면

root:/>lsfs -q
Name            Nodename   Mount Pt               VFS   Size    Options    Auto Accounting
/dev/hd4        --         /                      jfs   4194304 --         yes  no
  (lv size: 4194304, fs size: 4194304, frag size: 4096, nbpi: 2048, compress: no, bf: false, ag: 8)
/dev/hd1        --         /home                  jfs   121634816 --         yes  no
  (lv size: 121634816, fs size: 121634816, frag size: 512, nbpi: 4096, compress: no, bf: false, ag: 8)
/dev/hd2        --         /usr                   jfs   3932160 --         yes  no
  (lv size: 3932160, fs size: 3932160, frag size: 4096, nbpi: 4096, compress: no, bf: false, ag: 8)
/dev/hd9var     --         /var                   jfs   2097152 --         yes  no
  (lv size: 2097152, fs size: 2097152, frag size: 4096, nbpi: 4096, compress: no, bf: true, ag: 64)
/dev/hd3        --         /tmp                   jfs   2097152 --         yes  no
  (lv size: 2097152, fs size: 2097152, frag size: 4096, nbpi: 4096, compress: no, bf: false, ag: 8)
/proc           --         /proc                  procfs --      --         yes  no
/dev/hd10opt    --         /opt                   jfs   3932160 --         yes  no
  (lv size: 3932160, fs size: 3932160, frag size: 512, nbpi: 4096, compress: no, bf: false, ag: 8)

위의 bf가 false 일 경우 large file을 지원하지 않는다고 합니다.


반응형

+ Recent posts