所有网站域名都有,网站建设书籍目录,东莞网络优化专业乐云seo,重庆巫溪网站建设文章目录 前言一、apt 安装dbgsym vnlinux二、使用.ddeb包安装dbgsym vnlinux三、dbgsym发行版四、crash调试参考资料 前言
最近在适配 ubuntu系统#xff0c;记录一下其crash的安装。
一、apt 安装dbgsym vnlinux
# echo deb http://ddebs.ubuntu.com $(lsb_release… 文章目录 前言一、apt 安装dbgsym vnlinux二、使用.ddeb包安装dbgsym vnlinux三、dbgsym发行版四、crash调试参考资料 前言
最近在适配 ubuntu系统记录一下其crash的安装。
一、apt 安装dbgsym vnlinux
# echo deb http://ddebs.ubuntu.com $(lsb_release -cs) main restricted universe multiversedeb http://ddebs.ubuntu.com $(lsb_release -cs)-updates main restricted universe multiversedeb http://ddebs.ubuntu.com $(lsb_release -cs)-proposed main restricted universe multiverse | sudo tee -a /etc/apt/sources.list.d/ddebs.list$ sudo apt install ubuntu-dbgsym-keyring
$ sudo apt-get update# apt -y install linux-image-$(uname -r)-dbgsym
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:linux-image-unsigned-5.15.0-122-generic-dbgsym
The following NEW packages will be installed:linux-image-5.15.0-122-generic-dbgsym linux-image-unsigned-5.15.0-122-generic-dbgsym
0 upgraded, 2 newly installed, 0 to remove and 42 not upgraded.
Need to get 1,061 MB of archives.
After this operation, 6,917 MB of additional disk space will be used.
Get:1 http://ddebs.ubuntu.com jammy-updates/main amd64 linux-image-unsigned-5.15.0-122-generic-dbgsym amd64 5.15.0-122.132 [1,061 MB]二、使用.ddeb包安装dbgsym vnlinux
也可以直接在 http://ddebs.ubuntu.com/pool/main/l/linux/下载对应的.ddeb包
dpkg -i linux-image-unsigned-5.15.0-122-generic-dbgsym_5.15.0-122.132_amd64.ddeb三、dbgsym发行版
在http://ddebs.ubuntu.com/pool/main/l/linux/查看支持的dbgsym vmlinux
发行版内核版本14.043.13.016.044.4.018.044.15.020.045.4.022.045.15.024.046.8.0
其他内核版本请参考http://ddebs.ubuntu.com/pool/main/l/
四、crash调试
# cat /etc/os-release
PRETTY_NAMEUbuntu 22.04.1 LTS
NAMEUbuntu
VERSION_ID22.04
VERSION22.04.1 LTS (Jammy Jellyfish)
VERSION_CODENAMEjammy
IDubuntu
ID_LIKEdebian
HOME_URLhttps://www.ubuntu.com/
SUPPORT_URLhttps://help.ubuntu.com/
BUG_REPORT_URLhttps://bugs.launchpad.net/ubuntu/
PRIVACY_POLICY_URLhttps://www.ubuntu.com/legal/terms-and-policies/privacy-policy
UBUNTU_CODENAMEjammy# uname -r
6.5.0-18-generic我这里用ubuntu22.04 的6.5.0-18-generic内核调试为例
/** 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.** This is the only entry point used for 64-bit system calls. The* hardware interface is reasonably well designed and the register to* argument mapping Linux uses fits well with the registers that are* available when SYSCALL is used.** SYSCALL instructions can be found inlined in libc implementations as* well as some other programs and libraries. There are also a handful* of SYSCALL instructions in the vDSO used, for example, as a* clock_gettimeofday fallback.** 64-bit SYSCALL saves rip to rcx, clears rflags.RF, then saves rflags to r11,* then loads new ss, cs, and rip from previously programmed MSRs.* rflags gets masked by a value from another MSR (so CLD and CLAC* are not needed). SYSCALL does not save anything on the stack* and does not change rsp.** Registers on entry:* rax system call number* rcx return address* r11 saved rflags (note: r11 is callee-clobbered register in C ABI)* rdi arg0* rsi arg1* rdx arg2* r10 arg3 (needs to be moved to rcx to conform to C ABI)* r8 arg4* r9 arg5* (note: r12-r15, rbp, rbx are callee-preserved in C ABI)** Only called from user space.** When user can change pt_regs-foo always force IRET. That is because* it deals with uncanonical addresses better. SYSRET has trouble* with them due to bugs in both AMD and Intel CPUs.*/SYM_CODE_START(entry_SYSCALL_64)UNWIND_HINT_ENTRYENDBRswapgs/* tss.sp2 is scratch space. */movq %rsp, PER_CPU_VAR(cpu_tss_rw TSS_sp2)SWITCH_TO_KERNEL_CR3 scratch_reg%rspmovq PER_CPU_VAR(pcpu_hot X86_top_of_stack), %rspSYM_INNER_LABEL(entry_SYSCALL_64_safe_stack, SYM_L_GLOBAL)ANNOTATE_NOENDBR/* Construct struct pt_regs on stack */pushq $__USER_DS /* pt_regs-ss */pushq PER_CPU_VAR(cpu_tss_rw TSS_sp2) /* pt_regs-sp */pushq %r11 /* pt_regs-flags */pushq $__USER_CS /* pt_regs-cs */pushq %rcx /* pt_regs-ip */
SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)......SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)pushq %rax /* pt_regs-orig_ax */PUSH_AND_CLEAR_REGS rax$-ENOSYS/* IRQs are off. */movq %rsp, %rdi/* Sign extend the lower 32bit as syscall numbers are treated as int */movslq %eax, %rsi/* clobbers %rax, make sure it is after saving the syscall nr */IBRS_ENTERUNTRAIN_RETcall do_syscall_64 /* returns with IRQs disabled */......看其vmlinux cat /boot/System.map-6.5.0-18-generic | grep entry_SYSCALL_64_after_hwframe
ffffffff82200078 T entry_SYSCALL_64_after_hwframeffffffff82200078: 50 push %rax
ffffffff82200079: 57 push %rdi
ffffffff8220007a: 56 push %rsi
ffffffff8220007b: 52 push %rdx
ffffffff8220007c: 51 push %rcx
ffffffff8220007d: 6a da push $0xffffffffffffffda
ffffffff8220007f: 41 50 push %r8
ffffffff82200081: 41 51 push %r9
ffffffff82200083: 41 52 push %r10
ffffffff82200085: 41 53 push %r11
ffffffff82200087: 53 push %rbx
ffffffff82200088: 55 push %rbp
ffffffff82200089: 41 54 push %r12
ffffffff8220008b: 41 55 push %r13
ffffffff8220008d: 41 56 push %r14
ffffffff8220008f: 41 57 push %r15
ffffffff82200091: 31 f6 xor %esi,%esi
ffffffff82200093: 31 d2 xor %edx,%edx
......
ffffffff822000d1: 90 nop
ffffffff822000d2: 90 nop
ffffffff822000d3: 90 nop
ffffffff822000d4: 90 nop
ffffffff822000d5: 90 nop
ffffffff822000d6: 90 nop
ffffffff822000d7: 90 nop
ffffffff822000d8: 90 nop
ffffffff822000d9: 90 nop
ffffffff822000da: 90 nop
ffffffff822000db: 90 nop
ffffffff822000dc: 90 nop
ffffffff822000dd: 90 nop
ffffffff822000de: 90 nop
ffffffff822000df: 90 nop
ffffffff822000e0: 90 nop
ffffffff822000e1: e8 9a 4a f1 ff call 0xffffffff82114b80
ffffffff822000e6: 90 nop
ffffffff822000e7: 90 nop
ffffffff822000e8: 90 nop
ffffffff822000e9: 90 nop
ffffffff822000ea: 90 nop到了ffffffff822000d1处全是 nop指令
# cat /boot/System.map-6.5.0-18-generic | grep \do_syscall_64\
ffffffff82114b80 T do_syscall_64而我看crash
crash dis entry_SYSCALL_64_after_hwframe
0xffffffff97400078 entry_SYSCALL_64_after_hwframe: push %rax
0xffffffff97400079 entry_SYSCALL_64_after_hwframe1: push %rdi
0xffffffff9740007a entry_SYSCALL_64_after_hwframe2: push %rsi
0xffffffff9740007b entry_SYSCALL_64_after_hwframe3: push %rdx
0xffffffff9740007c entry_SYSCALL_64_after_hwframe4: push %rcx
0xffffffff9740007d entry_SYSCALL_64_after_hwframe5: push $0xffffffffffffffda
0xffffffff9740007f entry_SYSCALL_64_after_hwframe7: push %r8
0xffffffff97400081 entry_SYSCALL_64_after_hwframe9: push %r9
0xffffffff97400083 entry_SYSCALL_64_after_hwframe11: push %r10
0xffffffff97400085 entry_SYSCALL_64_after_hwframe13: push %r11
0xffffffff97400087 entry_SYSCALL_64_after_hwframe15: push %rbx
0xffffffff97400088 entry_SYSCALL_64_after_hwframe16: push %rbp
0xffffffff97400089 entry_SYSCALL_64_after_hwframe17: push %r12
0xffffffff9740008b entry_SYSCALL_64_after_hwframe19: push %r13
0xffffffff9740008d entry_SYSCALL_64_after_hwframe21: push %r14
0xffffffff9740008f entry_SYSCALL_64_after_hwframe23: push %r15
0xffffffff97400091 entry_SYSCALL_64_after_hwframe25: xor %esi,%esi
0xffffffff97400093 entry_SYSCALL_64_after_hwframe27: xor %edx,%edx
0xffffffff97400095 entry_SYSCALL_64_after_hwframe29: xor %ecx,%ecx
0xffffffff97400097 entry_SYSCALL_64_after_hwframe31: xor %r8d,%r8d
0xffffffff9740009a entry_SYSCALL_64_after_hwframe34: xor %r9d,%r9d
0xffffffff9740009d entry_SYSCALL_64_after_hwframe37: xor %r10d,%r10d
0xffffffff974000a0 entry_SYSCALL_64_after_hwframe40: xor %r11d,%r11d
0xffffffff974000a3 entry_SYSCALL_64_after_hwframe43: xor %ebx,%ebx
0xffffffff974000a5 entry_SYSCALL_64_after_hwframe45: xor %ebp,%ebp
0xffffffff974000a7 entry_SYSCALL_64_after_hwframe47: xor %r12d,%r12d
0xffffffff974000aa entry_SYSCALL_64_after_hwframe50: xor %r13d,%r13d
0xffffffff974000ad entry_SYSCALL_64_after_hwframe53: xor %r14d,%r14d
0xffffffff974000b0 entry_SYSCALL_64_after_hwframe56: xor %r15d,%r15d
0xffffffff974000b3 entry_SYSCALL_64_after_hwframe59: mov %rsp,%rdi
0xffffffff974000b6 entry_SYSCALL_64_after_hwframe62: movslq %eax,%rsi
0xffffffff974000b9 entry_SYSCALL_64_after_hwframe65: xchg %ax,%ax
0xffffffff974000bb entry_SYSCALL_64_after_hwframe67: mov $0x48,%ecx
0xffffffff974000c0 entry_SYSCALL_64_after_hwframe72: mov %gs:0x1fbd0,%rdx
0xffffffff974000c9 entry_SYSCALL_64_after_hwframe81: mov %edx,%eax
0xffffffff974000cb entry_SYSCALL_64_after_hwframe83: shr $0x20,%rdx
0xffffffff974000cf entry_SYSCALL_64_after_hwframe87: wrmsr
0xffffffff974000d1 entry_SYSCALL_64_after_hwframe89: jmp 0xffffffff974000e1 entry_SYSCALL_64_after_hwframe105
0xffffffff974000d3 entry_SYSCALL_64_after_hwframe91: int3
0xffffffff974000d4 entry_SYSCALL_64_after_hwframe92: int3
0xffffffff974000d5 entry_SYSCALL_64_after_hwframe93: int3
0xffffffff974000d6 entry_SYSCALL_64_after_hwframe94: int3
0xffffffff974000d7 entry_SYSCALL_64_after_hwframe95: int3
0xffffffff974000d8 entry_SYSCALL_64_after_hwframe96: int3
0xffffffff974000d9 entry_SYSCALL_64_after_hwframe97: int3
0xffffffff974000da entry_SYSCALL_64_after_hwframe98: int3
0xffffffff974000db entry_SYSCALL_64_after_hwframe99: int3
0xffffffff974000dc entry_SYSCALL_64_after_hwframe100: int3
0xffffffff974000dd entry_SYSCALL_64_after_hwframe101: int3
0xffffffff974000de entry_SYSCALL_64_after_hwframe102: int3
0xffffffff974000df entry_SYSCALL_64_after_hwframe103: int3
0xffffffff974000e0 entry_SYSCALL_64_after_hwframe104: int3
0xffffffff974000e1 entry_SYSCALL_64_after_hwframe105: call 0xffffffff97314b80 do_syscall_64在0xffffffff974000d1处nop指令变成了jmp
0xffffffff974000d1 entry_SYSCALL_64_after_hwframe89: jmp 0xffffffff974000e1 entry_SYSCALL_64_after_hwframe105
......
0xffffffff974000e1 entry_SYSCALL_64_after_hwframe105: call 0xffffffff97314b80 do_syscall_64与UNTRAIN_RET指令有关。
2
/** Save all registers in pt_regs. Return GSBASE related information* in EBX depending on the availability of the FSGSBASE instructions:** FSGSBASE R/EBX* N 0 - SWAPGS on exit* 1 - no SWAPGS on exit** Y GSBASE value at entry, must be restored in paranoid_exit** R14 - old CR3* R15 - old SPEC_CTRL*/
SYM_CODE_START(paranoid_entry)ANNOTATE_NOENDBRUNWIND_HINT_FUNCPUSH_AND_CLEAR_REGS save_ret1ENCODE_FRAME_POINTER 8/** Always stash CR3 in %r14. This value will be restored,* verbatim, at exit. Needed if paranoid_entry interrupted* another entry that already switched to the user CR3 value* but has not yet returned to userspace.** This is also why CS (stashed in the iret frame by the* hardware at entry) can not be used: this may be a return* to kernel code, but with a user CR3 value.** Switching CR3 does not depend on kernel GSBASE so it can* be done before switching to the kernel GSBASE. This is* required for FSGSBASE because the kernel GSBASE has to* be retrieved from a kernel internal table.*/SAVE_AND_SWITCH_TO_KERNEL_CR3 scratch_reg%rax save_reg%r14/** Handling GSBASE depends on the availability of FSGSBASE.** Without FSGSBASE the kernel enforces that negative GSBASE* values indicate kernel GSBASE. With FSGSBASE no assumptions* can be made about the GSBASE value when entering from user* space.*/ALTERNATIVE jmp .Lparanoid_entry_checkgs, , X86_FEATURE_FSGSBASE/** Read the current GSBASE and store it in %rbx unconditionally,* retrieve and set the current CPUs kernel GSBASE. The stored value* has to be restored in paranoid_exit unconditionally.** The unconditional write to GS base below ensures that no subsequent* loads based on a mispredicted GS base can happen, therefore no LFENCE* is needed here.*/SAVE_AND_SET_GSBASE scratch_reg%rax save_reg%rbxjmp .Lparanoid_gsbase_done.Lparanoid_entry_checkgs:/* EBX 1 - kernel GSBASE active, no restore required */movl $1, %ebx/** The kernel-enforced convention is a negative GSBASE indicates* a kernel value. No SWAPGS needed on entry and exit.*/movl $MSR_GS_BASE, %ecxrdmsrtestl %edx, %edxjs .Lparanoid_kernel_gsbase/* EBX 0 - SWAPGS required on exit */xorl %ebx, %ebxswapgs
.Lparanoid_kernel_gsbase:FENCE_SWAPGS_KERNEL_ENTRY
.Lparanoid_gsbase_done:/** Once we have CR3 and %GS setup save and set SPEC_CTRL. Just like* CR3 above, keep the old value in a callee saved register.*/IBRS_ENTER save_reg%r15UNTRAIN_RET_FROM_CALLRET
SYM_CODE_END(paranoid_entry)查看vmlinux
# cat /boot/System.map-6.5.0-18-generic | grep \paranoid_entry\
ffffffff822013b0 T paranoid_entryffffffff822013b0: 56 push %rsi
ffffffff822013b1: 48 8b 74 24 08 mov 0x8(%rsp),%rsi
ffffffff822013b6: 48 89 7c 24 08 mov %rdi,0x8(%rsp)
ffffffff822013bb: 52 push %rdx
ffffffff822013bc: 51 push %rcx
ffffffff822013bd: 50 push %rax
ffffffff822013be: 41 50 push %r8
ffffffff822013c0: 41 51 push %r9
ffffffff822013c2: 41 52 push %r10
ffffffff822013c4: 41 53 push %r11
ffffffff822013c6: 53 push %rbx
ffffffff822013c7: 55 push %rbp
ffffffff822013c8: 41 54 push %r12
ffffffff822013ca: 41 55 push %r13
ffffffff822013cc: 41 56 push %r14
ffffffff822013ce: 41 57 push %r15
ffffffff822013d0: 56 push %rsi
ffffffff822013d1: 31 f6 xor %esi,%esi
ffffffff822013d3: 31 d2 xor %edx,%edx
......
ffffffff82201484: 90 nop
ffffffff82201485: 90 nop
ffffffff82201486: 90 nop
ffffffff82201487: 90 nop
ffffffff82201488: 90 nop
ffffffff82201489: 90 nop
ffffffff8220148a: 90 nop
ffffffff8220148b: 90 nop
ffffffff8220148c: 90 nop
ffffffff8220148d: 90 nop
ffffffff8220148e: 90 nop
ffffffff8220148f: 90 nop
ffffffff82201490: 90 nop
ffffffff82201491: 90 nop
ffffffff82201492: 90 nop到了ffffffff82201484处都是nop指令。
crash查看
crash dis paranoid_entry
0xffffffff974013b0 paranoid_entry: push %rsi
0xffffffff974013b1 paranoid_entry1: mov 0x8(%rsp),%rsi
0xffffffff974013b6 paranoid_entry6: mov %rdi,0x8(%rsp)
0xffffffff974013bb paranoid_entry11: push %rdx
0xffffffff974013bc paranoid_entry12: push %rcx
0xffffffff974013bd paranoid_entry13: push %rax
0xffffffff974013be paranoid_entry14: push %r8
0xffffffff974013c0 paranoid_entry16: push %r9
0xffffffff974013c2 paranoid_entry18: push %r10
0xffffffff974013c4 paranoid_entry20: push %r11
0xffffffff974013c6 paranoid_entry22: push %rbx
0xffffffff974013c7 paranoid_entry23: push %rbp
0xffffffff974013c8 paranoid_entry24: push %r12
0xffffffff974013ca paranoid_entry26: push %r13
0xffffffff974013cc paranoid_entry28: push %r14
0xffffffff974013ce paranoid_entry30: push %r15
0xffffffff974013d0 paranoid_entry32: push %rsi
0xffffffff974013d1 paranoid_entry33: xor %esi,%esi
0xffffffff974013d3 paranoid_entry35: xor %edx,%edx
......
0xffffffff97401484 paranoid_entry212: jmp 0xffffffff97401493 paranoid_entry227
0xffffffff97401486 paranoid_entry214: int3
0xffffffff97401487 paranoid_entry215: int3
0xffffffff97401488 paranoid_entry216: int3
0xffffffff97401489 paranoid_entry217: int3
0xffffffff9740148a paranoid_entry218: int3
0xffffffff9740148b paranoid_entry219: int3
0xffffffff9740148c paranoid_entry220: int3
0xffffffff9740148d paranoid_entry221: int3
0xffffffff9740148e paranoid_entry222: int3
0xffffffff9740148f paranoid_entry223: int3
0xffffffff97401490 paranoid_entry224: int3
0xffffffff97401491 paranoid_entry225: int3
0xffffffff97401492 paranoid_entry226: int3
0xffffffff97401493 paranoid_entry227: ret
0xffffffff97401494 paranoid_entry228: int3
0xffffffff97401495 paranoid_entry229: int3
0xffffffff97401496 paranoid_entry230: int3
0xffffffff97401497 paranoid_entry231: int3到了0xffffffff97401484
0xffffffff97401484 paranoid_entry212: jmp 0xffffffff97401493 paranoid_entry227
0xffffffff97401493 paranoid_entry227: ret也是nop指令变成了jmp指令。 这与UNTRAIN_RET_FROM_CALL指令有关。
参考资料
https://ebpf.top/post/ubuntu_kdump_crash/
