Return to Question

root@hostname:~# dmesg | tail -n 200 [ 223.850430] XFS (dm-12): Unmounting Filesystem [ 247.989713] XFS (dm-12): Mounting V5 Filesystem [ 248.014825] XFS (dm-12): Ending clean mount [ 248.021340] xfs filesystem being mounted at /pg_data supports timestamps until 2038-01-19 (0x7fffffff) [ 263.061620] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 263.062166] XFS (dm-12): Unmount and run xfs_repair [ 263.062439] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 263.062686] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03 ............(... [ 263.062951] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00 . . ......b...b. [ 263.063191] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00 X.b. .b...b...b. [ 263.063456] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00 [email protected]. [ 263.063805] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00 ..b..b.(.b...b. [ 263.064105] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00 ..b...b.H.b...b. [ 263.064385] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00 ..b...b.h.b.0.b. [ 263.064690] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00 ..b...b...b.P.b. [ 263.064978] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117 [ 263.066864] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0xbb7bbf40 xfs_inode_buf_verify [ 263.067526] XFS (dm-12): Unmount and run xfs_repair [ 263.067840] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 263.068179] 00000000: e4 11 00 00 00 ad e4 dc 00 00 00 00 98 00 00 01 ................ [ 263.068491] 00000010: 00 20 04 20 00 00 00 00 08 9f e4 01 10 9e e4 01 . . ............ [ 263.068815] 00000020: 18 9d e4 01 20 9c e4 01 28 9b e4 01 30 9a e4 01 .... ...(...0... [ 263.069209] 00000030: 38 99 e4 01 40 98 e4 01 48 97 e4 01 50 96 e4 01 [email protected]... [ 263.069501] 00000040: 58 95 e4 01 60 94 e4 01 68 93 e4 01 70 92 e4 01 X......h...p... [ 263.069865] 00000050: 78 91 e4 01 80 90 e4 01 88 8f e4 01 90 8e e4 01 x............... [ 263.070199] 00000060: 98 8d e4 01 a0 8c e4 01 a8 8b e4 01 b0 8a e4 01 ................ [ 263.070552] 00000070: b8 89 e4 01 c0 88 e4 01 c8 87 e4 01 d0 86 e4 01 ................ [ 263.070868] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0xbb7bbf40 len 32 error 117 [ 263.073850] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 263.074615] XFS (dm-12): Unmount and run xfs_repair [ 263.074954] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 263.075312] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03 ............(... [ 263.075661] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00 . . ......b...b. [ 263.076014] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00 X.b. .b...b...b. [ 263.076398] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00 [email protected]. [ 263.076773] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00 ..b..b.(.b...b. [ 263.077116] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00 ..b...b.H.b...b. [ 263.077564] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00 ..b...b.h.b.0.b. [ 263.077918] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00 ..b...b...b.P.b. [ 263.078278] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117 [ 267.928729] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 267.929699] XFS (dm-12): Unmount and run xfs_repair [ 267.930128] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 267.930498] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03 ............(... [ 267.930880] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00 . . ......b...b. [ 267.931288] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00 X.b. .b...b...b. [ 267.931651] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00 [email protected]. [ 267.932058] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00 ..b..b.(.b...b. [ 267.932398] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00 ..b...b.H.b...b. [ 267.932727] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00 ..b...b.h.b.0.b. [ 267.933051] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00 ..b...b...b.P.b. [ 267.933419] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117 [ 267.935092] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0xbb7bbf40 xfs_inode_buf_verify [ 267.935855] XFS (dm-12): Unmount and run xfs_repair [ 267.936222] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 267.936591] 00000000: e4 11 00 00 00 ad e4 dc 00 00 00 00 98 00 00 01 ................ [ 267.936946] 00000010: 00 20 04 20 00 00 00 00 08 9f e4 01 10 9e e4 01 . . ............ [ 267.937405] 00000020: 18 9d e4 01 20 9c e4 01 28 9b e4 01 30 9a e4 01 .... ...(...0... [ 267.937863] 00000030: 38 99 e4 01 40 98 e4 01 48 97 e4 01 50 96 e4 01 [email protected]... [ 267.938294] 00000040: 58 95 e4 01 60 94 e4 01 68 93 e4 01 70 92 e4 01 X......h...p... [ 267.938645] 00000050: 78 91 e4 01 80 90 e4 01 88 8f e4 01 90 8e e4 01 x............... [ 267.938986] 00000060: 98 8d e4 01 a0 8c e4 01 a8 8b e4 01 b0 8a e4 01 ................ [ 267.939323] 00000070: b8 89 e4 01 c0 88 e4 01 c8 87 e4 01 d0 86 e4 01 ................ [ 267.939701] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0xbb7bbf40 len 32 error 117 [ 267.941264] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 267.942026] XFS (dm-12): Unmount and run xfs_repair [ 267.942417] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 267.942832] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03 ............(... [ 267.943224] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00 . . ......b...b. [ 267.943636] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00 X.b. .b...b...b. [ 267.944066] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00 [email protected]. [ 267.944426] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00 ..b..b.(.b...b. [ 267.944791] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00 ..b...b.H.b...b. [ 267.945137] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00 ..b...b.h.b.0.b. [ 267.945527] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00 ..b...b...b.P.b. [ 267.945897] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117 [ 273.543750] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 273.544828] XFS (dm-12): Unmount and run xfs_repair

root@hostname:~# dmesg | tail -n 200
[  223.850430] XFS (dm-12): Unmounting Filesystem
[  247.989713] XFS (dm-12): Mounting V5 Filesystem
[  248.014825] XFS (dm-12): Ending clean mount
[  248.021340] xfs filesystem being mounted at /pg_data supports timestamps until 2038-01-19 (0x7fffffff)
[  263.061620] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify
[  263.062166] XFS (dm-12): Unmount and run xfs_repair
[  263.062439] XFS (dm-12): First 128 bytes of corrupted metadata buffer:
[  263.062686] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03  ............(...
[  263.062951] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00  . . ......b...b.
[  263.063191] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00  X.b. .b...b...b.
[  263.063456] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00  [email protected].
[  263.063805] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00  ..b.`.b.(.b...b.
[  263.064105] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00  ..b...b.H.b...b.
[  263.064385] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00  ..b...b.h.b.0.b.
[  263.064690] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00  ..b...b...b.P.b.
[  263.064978] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117
[  263.066864] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0xbb7bbf40 xfs_inode_buf_verify
[  263.067526] XFS (dm-12): Unmount and run xfs_repair
[  263.067840] XFS (dm-12): First 128 bytes of corrupted metadata buffer:
[  263.068179] 00000000: e4 11 00 00 00 ad e4 dc 00 00 00 00 98 00 00 01  ................
[  263.068491] 00000010: 00 20 04 20 00 00 00 00 08 9f e4 01 10 9e e4 01  . . ............
[  263.068815] 00000020: 18 9d e4 01 20 9c e4 01 28 9b e4 01 30 9a e4 01  .... ...(...0...
[  263.069209] 00000030: 38 99 e4 01 40 98 e4 01 48 97 e4 01 50 96 e4 01  [email protected]...
[  263.069501] 00000040: 58 95 e4 01 60 94 e4 01 68 93 e4 01 70 92 e4 01  X...`...h...p...
[  263.069865] 00000050: 78 91 e4 01 80 90 e4 01 88 8f e4 01 90 8e e4 01  x...............
[  263.070199] 00000060: 98 8d e4 01 a0 8c e4 01 a8 8b e4 01 b0 8a e4 01  ................
[  263.070552] 00000070: b8 89 e4 01 c0 88 e4 01 c8 87 e4 01 d0 86 e4 01  ................
[  263.070868] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0xbb7bbf40 len 32 error 117
[  263.073850] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify
[  263.074615] XFS (dm-12): Unmount and run xfs_repair
[  263.074954] XFS (dm-12): First 128 bytes of corrupted metadata buffer:
[  263.075312] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03  ............(...
[  263.075661] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00  . . ......b...b.
[  263.076014] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00  X.b. .b...b...b.
[  263.076398] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00  [email protected].
[  263.076773] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00  ..b.`.b.(.b...b.
[  263.077116] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00  ..b...b.H.b...b.
[  263.077564] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00  ..b...b.h.b.0.b.
[  263.077918] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00  ..b...b...b.P.b.
[  263.078278] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117
[  267.928729] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify
[  267.929699] XFS (dm-12): Unmount and run xfs_repair
[  267.930128] XFS (dm-12): First 128 bytes of corrupted metadata buffer:
[  267.930498] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03  ............(...
[  267.930880] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00  . . ......b...b.
[  267.931288] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00  X.b. .b...b...b.
[  267.931651] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00  [email protected].
[  267.932058] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00  ..b.`.b.(.b...b.
[  267.932398] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00  ..b...b.H.b...b.
[  267.932727] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00  ..b...b.h.b.0.b.
[  267.933051] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00  ..b...b...b.P.b.
[  267.933419] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117
[  267.935092] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0xbb7bbf40 xfs_inode_buf_verify
[  267.935855] XFS (dm-12): Unmount and run xfs_repair
[  267.936222] XFS (dm-12): First 128 bytes of corrupted metadata buffer:
[  267.936591] 00000000: e4 11 00 00 00 ad e4 dc 00 00 00 00 98 00 00 01  ................
[  267.936946] 00000010: 00 20 04 20 00 00 00 00 08 9f e4 01 10 9e e4 01  . . ............
[  267.937405] 00000020: 18 9d e4 01 20 9c e4 01 28 9b e4 01 30 9a e4 01  .... ...(...0...
[  267.937863] 00000030: 38 99 e4 01 40 98 e4 01 48 97 e4 01 50 96 e4 01  [email protected]...
[  267.938294] 00000040: 58 95 e4 01 60 94 e4 01 68 93 e4 01 70 92 e4 01  X...`...h...p...
[  267.938645] 00000050: 78 91 e4 01 80 90 e4 01 88 8f e4 01 90 8e e4 01  x...............
[  267.938986] 00000060: 98 8d e4 01 a0 8c e4 01 a8 8b e4 01 b0 8a e4 01  ................
[  267.939323] 00000070: b8 89 e4 01 c0 88 e4 01 c8 87 e4 01 d0 86 e4 01  ................
[  267.939701] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0xbb7bbf40 len 32 error 117
[  267.941264] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify
[  267.942026] XFS (dm-12): Unmount and run xfs_repair
[  267.942417] XFS (dm-12): First 128 bytes of corrupted metadata buffer:
[  267.942832] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03  ............(...
[  267.943224] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00  . . ......b...b.
[  267.943636] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00  X.b. .b...b...b.
[  267.944066] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00  [email protected].
[  267.944426] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00  ..b.`.b.(.b...b.
[  267.944791] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00  ..b...b.H.b...b.
[  267.945137] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00  ..b...b.h.b.0.b.
[  267.945527] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00  ..b...b...b.P.b.
[  267.945897] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117
[  273.543750] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify
[  273.544828] XFS (dm-12): Unmount and run xfs_repair

We have an Ubuntu 22.04 server on which there is a Postgres database. During the after done (afternoon?? After patching the kernel??) the kernel patch team rebooted the server and the database crashed. This is the second time we have faced this issue, and the database team has not made any database backup, but they have shut down the database services properly.

root@hostname:~# dmesg | tail -n 200 [ 223.850430] XFS (dm-12): Unmounting Filesystem [ 247.989713] XFS (dm-12): Mounting V5 Filesystem [ 248.014825] XFS (dm-12): Ending clean mount [ 248.021340] xfs filesystem being mounted at /pg_data supports timestamps until 2038-01-19 (0x7fffffff) [ 263.061620] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 263.062166] XFS (dm-12): Unmount and run xfs_repair [ 263.062439] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 263.062686] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03 ............(... [ 263.062951] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00 . . ......b...b. [ 263.063191] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00 X.b. .b...b...b. [ 263.063456] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00 [email protected]. [ 263.063805] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00 ..b..b.(.b...b. [ 263.064105] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00 ..b...b.H.b...b. [ 263.064385] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00 ..b...b.h.b.0.b. [ 263.064690] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00 ..b...b...b.P.b. [ 263.064978] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117 [ 263.066864] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0xbb7bbf40 xfs_inode_buf_verify [ 263.067526] XFS (dm-12): Unmount and run xfs_repair [ 263.067840] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 263.068179] 00000000: e4 11 00 00 00 ad e4 dc 00 00 00 00 98 00 00 01 ................ [ 263.068491] 00000010: 00 20 04 20 00 00 00 00 08 9f e4 01 10 9e e4 01 . . ............ [ 263.068815] 00000020: 18 9d e4 01 20 9c e4 01 28 9b e4 01 30 9a e4 01 .... ...(...0... [ 263.069209] 00000030: 38 99 e4 01 40 98 e4 01 48 97 e4 01 50 96 e4 01 [email protected]... [ 263.069501] 00000040: 58 95 e4 01 60 94 e4 01 68 93 e4 01 70 92 e4 01 X......h...p... [ 263.069865] 00000050: 78 91 e4 01 80 90 e4 01 88 8f e4 01 90 8e e4 01 x............... [ 263.070199] 00000060: 98 8d e4 01 a0 8c e4 01 a8 8b e4 01 b0 8a e4 01 ................ [ 263.070552] 00000070: b8 89 e4 01 c0 88 e4 01 c8 87 e4 01 d0 86 e4 01 ................ [ 263.070868] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0xbb7bbf40 len 32 error 117 [ 263.073850] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 263.074615] XFS (dm-12): Unmount and run xfs_repair [ 263.074954] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 263.075312] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03 ............(... [ 263.075661] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00 . . ......b...b. [ 263.076014] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00 X.b. .b...b...b. [ 263.076398] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00 [email protected]. [ 263.076773] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00 ..b..b.(.b...b. [ 263.077116] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00 ..b...b.H.b...b. [ 263.077564] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00 ..b...b.h.b.0.b. [ 263.077918] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00 ..b...b...b.P.b. [ 263.078278] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117 [ 267.928729] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 267.929699] XFS (dm-12): Unmount and run xfs_repair [ 267.930128] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 267.930498] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03 ............(... [ 267.930880] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00 . . ......b...b. [ 267.931288] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00 X.b. .b...b...b. [ 267.931651] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00 [email protected]. [ 267.932058] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00 ..b..b.(.b...b. [ 267.932398] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00 ..b...b.H.b...b. [ 267.932727] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00 ..b...b.h.b.0.b. [ 267.933051] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00 ..b...b...b.P.b. [ 267.933419] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117 [ 267.935092] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0xbb7bbf40 xfs_inode_buf_verify [ 267.935855] XFS (dm-12): Unmount and run xfs_repair [ 267.936222] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 267.936591] 00000000: e4 11 00 00 00 ad e4 dc 00 00 00 00 98 00 00 01 ................ [ 267.936946] 00000010: 00 20 04 20 00 00 00 00 08 9f e4 01 10 9e e4 01 . . ............ [ 267.937405] 00000020: 18 9d e4 01 20 9c e4 01 28 9b e4 01 30 9a e4 01 .... ...(...0... [ 267.937863] 00000030: 38 99 e4 01 40 98 e4 01 48 97 e4 01 50 96 e4 01 [email protected]... [ 267.938294] 00000040: 58 95 e4 01 60 94 e4 01 68 93 e4 01 70 92 e4 01 X......h...p... [ 267.938645] 00000050: 78 91 e4 01 80 90 e4 01 88 8f e4 01 90 8e e4 01 x............... [ 267.938986] 00000060: 98 8d e4 01 a0 8c e4 01 a8 8b e4 01 b0 8a e4 01 ................ [ 267.939323] 00000070: b8 89 e4 01 c0 88 e4 01 c8 87 e4 01 d0 86 e4 01 ................ [ 267.939701] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0xbb7bbf40 len 32 error 117 [ 267.941264] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 267.942026] XFS (dm-12): Unmount and run xfs_repair [ 267.942417] XFS (dm-12): First 128 bytes of corrupted metadata buffer: [ 267.942832] 00000000: e5 11 00 00 88 f5 89 06 00 00 05 00 28 02 c8 03 ............(... [ 267.943224] 00000010: 00 20 04 20 00 00 00 00 c8 9f 62 00 90 9f 62 00 . . ......b...b. [ 267.943636] 00000020: 58 9f 62 00 20 9f 62 00 e8 9e 62 00 b0 9e 62 00 X.b. .b...b...b. [ 267.944066] 00000030: 78 9e 62 00 40 9e 62 00 08 9e 62 00 d0 9d 62 00 [email protected]. [ 267.944426] 00000040: 98 9d 62 00 60 9d 62 00 28 9d 62 00 f0 9c 62 00 ..b..b.(.b...b. [ 267.944791] 00000050: b8 9c 62 00 80 9c 62 00 48 9c 62 00 10 9c 62 00 ..b...b.H.b...b. [ 267.945137] 00000060: d8 9b 62 00 a0 9b 62 00 68 9b 62 00 30 9b 62 00 ..b...b.h.b.0.b. [ 267.945527] 00000070: f8 9a 62 00 c0 9a 62 00 88 9a 62 00 50 9a 62 00 ..b...b...b.P.b. [ 267.945897] XFS (dm-12): metadata I/O error in "xfs_imap_to_bp+0x4f/0x70 [xfs]" at daddr 0x9c383240 len 32 error 117 [ 273.543750] XFS (dm-12): Metadata corruption detected at xfs_inode_buf_verify+0x15d/0x170 [xfs], xfs_inode block 0x9c383240 xfs_inode_buf_verify [ 273.544828] XFS (dm-12): Unmount and run xfs_repair

We have an Ubuntu 22.04 server on which there is a Postgres database. During the after done (After patching the kernel) the kernel patch team rebooted the server and the database crashed. This is the second time we have faced this issue, and the database team has not made any database backup, but they have shut down the database services properly.

We have an Ubuntu 22.04 server on which there is a Postgres database. During the after done (afternoon?? After patching the kernel??) the kernel patch team rebooted the server and the database crashed. This is the second time we have faced this issue, and the database team has not made any database backup, but they have shut down the database services properly.

Now I want to find out why the database got corrupted again and again. Previously I had suggested making a full database backup before the kernel patch and back it up both times. They are relying on a vnsnapshot (VM snapshot??) only.

Now they are asking me for a root cause analysis.

Please help me to solve this problem. I want to identify why it's getting corrupted again and again. Is it mandatory to back up the database during kernel patch activity, so I can clearly tell them to back up the database next time (at some other time??)?

We have an Ubuntu 22.04 server on which there is a Postgres database. During the after done (afternoon?? After patching the kernel??) the kernel patch team rebooted the server and the database crashed. This is the second time we have faced this issue, and the database team has not made any database backup, but they have shut down the database services properly.

Now I want to find out why the database got corrupted again and again. Previously I had suggested making a full database backup before the kernel patch and backing it up both times. They are relying on a vnsnapshot (VM snapshot??) only.

Now they are asking me for a root cause analysis.

Please help me to solve this problem. I want to identify why it's getting corrupted again and again. Is it mandatory to back up the database during kernel patch activity, so I can tell them to back up the database next time (at a different time??)?