Skip to content

Commit

Permalink
[Show] Update the subcommands of Kdump. (sonic-net#1682)
Browse files Browse the repository at this point in the history 
Signed-off-by: Yong Zhao yozhao@microsoft.com

What I did
This PR fixed the following issues:

Initially we need add prefix sudo when ran the show command of Kdump. This is unnecessary.

I re-organized the logic of implementing show command and also update the script sonic-kdump-config.

How I did it
Since the script sonic-kdump-config can only be executed by root user, I moved some functions from this file to
the kdump.py such that we do not need run the show command with sudo.

How to verify it
I verified this change on the DuT str-msn2700-03.
  • Loading branch information
@yozhao101
yozhao101 authored Aug 22, 2021
1 parent c03342f commit 4cb3b72
Show file tree
Hide file tree
Showing 3 changed files with 290 additions and 169 deletions.
116 changes: 116 additions & 0 deletions doc/Command-Reference.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@
* [Kubernetes](#Kubernetes)
* [Kubernetes show commands](#Kubernetes-show-commands)
* [Kubernetes config commands](#Kubernetes-config-commands)
* [Linux Kernel Dump](#kdump)
* [Linux Kernel Dump show commands](#kdump-show-commands)
* [Linux Kernel Dump config commands](#kdump-config-commands)
* [LLDP](#lldp)
* [LLDP show commands](#lldp-show-commands)
* [Loading, Reloading And Saving Configuration](#loading-reloading-and-saving-configuration)
Expand Down Expand Up @@ -4538,6 +4541,119 @@ This command displays the kubernetes server status.
----------- ------ ----------- -------------------
10.3.157.24 6443 true 2020-11-15 18:25:05
```
Go Back To [Beginning of the document](#) or [Beginning of this section](#Kubernetes)

## Linux Kernel Dump

This section demonstrates the show commands and configuration commands of Linux kernel dump mechanism in SONiC.

### Linux Kernel Dump show commands

**show kdump config**

This command shows the configuration of Linux kernel dump.

- Usage:
```
show kdump config
```

- Example:
```
admin@sonic:$ show kdump config
Kdump administrative mode: Disabled
Kdump operational mode: Unready
Kdump memory researvation: 0M-2G:256M,2G-4G:320M,4G-8G:384M,8G-:448M
Maximum number of Kdump files: 3
```

**show kdump files**

This command shows the Linux kernel core dump files and dmesg files which are
generated by kernel dump tool.

- Usage:
```
show kdump files
```

- Example:
```
admin@sonic:~$ show kdump files
Kernel core dump files Kernel dmesg files
------------------------------------------ ------------------------------------------
/var/crash/202106242344/kdump.202106242344 /var/crash/202106242344/dmesg.202106242344
/var/crash/202106242337/kdump.202106242337 /var/crash/202106242337/dmesg.202106242337
```

**show kdump logging <file_name> <num_of_lines>**

By default, this command will show the last 10 lines of latest dmesg file.
This command can also accept a specific file name and number of lines as arguments.

- Usage:
```
show kdump logging
```

- Example:
```
admin@sonic:~$ show kdump logging
[ 157.642053] RSP: 002b:00007fff1beee708 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 157.732635] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fc3887d4504
[ 157.818015] RDX: 0000000000000002 RSI: 000055d388eceb40 RDI: 0000000000000001
[ 157.903401] RBP: 000055d388eceb40 R08: 000000000000000a R09: 00007fc3888255f0
[ 157.988784] R10: 000000000000000a R11: 0000000000000246 R12: 00007fc3888a6760
[ 158.074166] R13: 0000000000000002 R14: 00007fc3888a1760 R15: 0000000000000002
[ 158.159553] Modules linked in: nft_chain_route_ipv6(E) nft_chain_route_ipv4(E) xt_TCPMSS(E) dummy(E) team_mode_loadbalance(E) team(E) sx_bfd(OE) sx_netdev(OE) psample(E) sx_core(OE) 8021q(E) garp(E) mrp(E) mst_pciconf(OE) mst_pci(OE) xt_hl(E) xt_tcpudp(E) ip6_tables(E) nft_compat(E) nft_chain_nat_ipv4(E) nf_nat_ipv4(E) nft_counter(E) xt_conntrack(E) nf_nat(E) jc42(E) nf_conntrack_netlink(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) xfrm_user(E) xfrm_algo(E) mlxsw_minimal(E) mlxsw_i2c(E) i2c_mux_reg(E) i2c_mux(E) i2c_mlxcpld(E) leds_mlxreg(E) mlxreg_io(E) mlxreg_hotplug(E) mei_wdt(E) evdev(E) intel_rapl(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) kvm_intel(E) mlx_platform(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) ghash_clmulni_intel(E) intel_cstate(E) intel_uncore(E)
[ 159.016731] intel_rapl_perf(E) pcspkr(E) sg(E) iTCO_wdt(E) iTCO_vendor_support(E) mei_me(E) mei(E) bonding(E) pcc_cpufreq(E) video(E) button(E) ebt_vlan(E) ebtable_broute(E) bridge(E) stp(E) llc(E) ebtable_nat(E) ebtable_filter(E) ebtables(E) nf_tables(E) nfnetlink(E) xdpe12284(E) at24(E) ledtrig_timer(E) tmp102(E) lm75(E) drm(E) coretemp(E) max1363(E) industrialio_triggered_buffer(E) kfifo_buf(E) industrialio(E) tps53679(E) fuse(E) pmbus(E) pmbus_core(E) i2c_dev(E) configfs(E) ip_tables(E) x_tables(E) autofs4(E) loop(E) ext4(E) crc16(E) mbcache(E) jbd2(E) crc32c_generic(E) fscrypto(E) ecb(E) crypto_simd(E) cryptd(E) glue_helper(E) aes_x86_64(E) nvme(E) nvme_core(E) nls_utf8(E) nls_cp437(E) nls_ascii(E) vfat(E) fat(E) overlay(E) squashfs(E) zstd_decompress(E) xxhash(E) sd_mod(E) gpio_ich(E) ahci(E)
[ 159.864532] libahci(E) mlxsw_core(E) devlink(E) ehci_pci(E) ehci_hcd(E) crc32c_intel(E) libata(E) i2c_i801(E) scsi_mod(E) usbcore(E) usb_common(E) lpc_ich(E) mfd_core(E) e1000e(E) fan(E) thermal(E)
[ 160.075846] CR2: 0000000000000000
```
You can specify a file name in order to show its
last 10 lines.

- Example:
```
admin@sonic:~$ show kdump logging dmesg.202106242337
[ 654.120195] RSP: 002b:00007ffe697690f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 654.210778] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fcfca27b504
[ 654.296157] RDX: 0000000000000002 RSI: 000055a6e4d1b3f0 RDI: 0000000000000001
[ 654.381543] RBP: 000055a6e4d1b3f0 R08: 000000000000000a R09: 00007fcfca2cc5f0
[ 654.466925] R10: 000000000000000a R11: 0000000000000246 R12: 00007fcfca34d760
[ 654.552310] R13: 0000000000000002 R14: 00007fcfca348760 R15: 0000000000000002
[ 654.637694] Modules linked in: binfmt_misc(E) nft_chain_route_ipv6(E) nft_chain_route_ipv4(E) xt_TCPMSS(E) dummy(E) team_mode_loadbalance(E) team(E) sx_bfd(OE) sx_netdev(OE) psample(E) sx_core(OE) 8021q(E) garp(E) mrp(E) mst_pciconf(OE) mst_pci(OE) xt_hl(E) xt_tcpudp(E) ip6_tables(E) nft_chain_nat_ipv4(E) nf_nat_ipv4(E) nft_compat(E) nft_counter(E) xt_conntrack(E) nf_nat(E) jc42(E) nf_conntrack_netlink(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) xfrm_user(E) xfrm_algo(E) mlxsw_minimal(E) mlxsw_i2c(E) i2c_mux_reg(E) i2c_mux(E) mlxreg_hotplug(E) mlxreg_io(E) i2c_mlxcpld(E) leds_mlxreg(E) mei_wdt(E) evdev(E) intel_rapl(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) kvm_intel(E) kvm(E) mlx_platform(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) ghash_clmulni_intel(E) intel_cstate(E)
[ 655.493833] intel_uncore(E) intel_rapl_perf(E) pcspkr(E) sg(E) iTCO_wdt(E) iTCO_vendor_support(E) mei_me(E) mei(E) bonding(E) video(E) button(E) pcc_cpufreq(E) ebt_vlan(E) ebtable_broute(E) bridge(E) stp(E) llc(E) ebtable_nat(E) ebtable_filter(E) ebtables(E) nf_tables(E) nfnetlink(E) xdpe12284(E) at24(E) ledtrig_timer(E) tmp102(E) drm(E) lm75(E) coretemp(E) max1363(E) industrialio_triggered_buffer(E) kfifo_buf(E) industrialio(E) fuse(E) tps53679(E) pmbus(E) pmbus_core(E) i2c_dev(E) configfs(E) ip_tables(E) x_tables(E) autofs4(E) loop(E) ext4(E) crc16(E) mbcache(E) jbd2(E) crc32c_generic(E) fscrypto(E) ecb(E) crypto_simd(E) cryptd(E) glue_helper(E) aes_x86_64(E) nvme(E) nvme_core(E) nls_utf8(E) nls_cp437(E) nls_ascii(E) vfat(E) fat(E) overlay(E) squashfs(E) zstd_decompress(E) xxhash(E) sd_mod(E)
[ 656.337476] gpio_ich(E) ahci(E) mlxsw_core(E) libahci(E) devlink(E) crc32c_intel(E) libata(E) i2c_i801(E) scsi_mod(E) lpc_ich(E) mfd_core(E) ehci_pci(E) ehci_hcd(E) usbcore(E) e1000e(E) usb_common(E) fan(E) thermal(E)
[ 656.569590] CR2: 0000000000000000
```
You can also specify a file name and number of lines in order to show the
last number of lines.

- Example:
```
admin@sonic:~$ show kdump logging dmesg.202106242337 -l 20
[ 653.525427] __handle_sysrq.cold.9+0x45/0xf2
[ 653.576487] write_sysrq_trigger+0x2b/0x30
[ 653.625472] proc_reg_write+0x39/0x60
[ 653.669252] vfs_write+0xa5/0x1a0
[ 653.708881] ksys_write+0x57/0xd0
[ 653.748501] do_syscall_64+0x53/0x110
[ 653.792287] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 653.852707] RIP: 0033:0x7fcfca27b504
[ 653.895452] Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 48 8d 05 f9 61 0d 00 8b 00 85 c0 75 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 49 89 d4 55 48 89 f5 53
[ 654.120195] RSP: 002b:00007ffe697690f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 654.210778] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fcfca27b504
[ 654.296157] RDX: 0000000000000002 RSI: 000055a6e4d1b3f0 RDI: 0000000000000001
[ 654.381543] RBP: 000055a6e4d1b3f0 R08: 000000000000000a R09: 00007fcfca2cc5f0
[ 654.466925] R10: 000000000000000a R11: 0000000000000246 R12: 00007fcfca34d760
[ 654.552310] R13: 0000000000000002 R14: 00007fcfca348760 R15: 0000000000000002
[ 654.637694] Modules linked in: binfmt_misc(E) nft_chain_route_ipv6(E) nft_chain_route_ipv4(E) xt_TCPMSS(E) dummy(E) team_mode_loadbalance(E) team(E) sx_bfd(OE) sx_netdev(OE) psample(E) sx_core(OE) 8021q(E) garp(E) mrp(E) mst_pciconf(OE) mst_pci(OE) xt_hl(E) xt_tcpudp(E) ip6_tables(E) nft_chain_nat_ipv4(E) nf_nat_ipv4(E) nft_compat(E) nft_counter(E) xt_conntrack(E) nf_nat(E) jc42(E) nf_conntrack_netlink(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) xfrm_user(E) xfrm_algo(E) mlxsw_minimal(E) mlxsw_i2c(E) i2c_mux_reg(E) i2c_mux(E) mlxreg_hotplug(E) mlxreg_io(E) i2c_mlxcpld(E) leds_mlxreg(E) mei_wdt(E) evdev(E) intel_rapl(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) kvm_intel(E) kvm(E) mlx_platform(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) ghash_clmulni_intel(E) intel_cstate(E)
[ 655.493833] intel_uncore(E) intel_rapl_perf(E) pcspkr(E) sg(E) iTCO_wdt(E) iTCO_vendor_support(E) mei_me(E) mei(E) bonding(E) video(E) button(E) pcc_cpufreq(E) ebt_vlan(E) ebtable_broute(E) bridge(E) stp(E) llc(E) ebtable_nat(E) ebtable_filter(E) ebtables(E) nf_tables(E) nfnetlink(E) xdpe12284(E) at24(E) ledtrig_timer(E) tmp102(E) drm(E) lm75(E) coretemp(E) max1363(E) industrialio_triggered_buffer(E) kfifo_buf(E) industrialio(E) fuse(E) tps53679(E) pmbus(E) pmbus_core(E) i2c_dev(E) configfs(E) ip_tables(E) x_tables(E) autofs4(E) loop(E) ext4(E) crc16(E) mbcache(E) jbd2(E) crc32c_generic(E) fscrypto(E) ecb(E) crypto_simd(E) cryptd(E) glue_helper(E) aes_x86_64(E) nvme(E) nvme_core(E) nls_utf8(E) nls_cp437(E) nls_ascii(E) vfat(E) fat(E) overlay(E) squashfs(E) zstd_decompress(E) xxhash(E) sd_mod(E)
[ 656.337476] gpio_ich(E) ahci(E) mlxsw_core(E) libahci(E) devlink(E) crc32c_intel(E) libata(E) i2c_i801(E) scsi_mod(E) lpc_ich(E) mfd_core(E) ehci_pci(E) ehci_hcd(E) usbcore(E) e1000e(E) usb_common(E) fan(E) thermal(E)
[ 656.569590] CR2: 0000000000000000
```
Go Back To [Beginning of the document](#) or [Beginning of this section](#kdump)

## LLDP

Expand Down
99 changes: 0 additions & 99 deletions scripts/sonic-kdump-config
View file
Original file line number Diff line number Diff line change
Expand Up @@ -607,86 +607,6 @@ def cmd_kdump_num_dumps(verbose, num_dumps):
kdump_memory = get_kdump_memory()
save_config(kdump_enabled, kdump_memory, num_dumps)

## Command: Display kdump status
def cmd_kdump_status():
print('Kdump Administrative Mode: ', end='')
kdump_enabled = get_kdump_administrative_mode()
if kdump_enabled:
print('Enabled')
else:
print('Disabled')

print('Kdump Operational State: ', end='')
(rc, lines, err_str) = run_command("/usr/sbin/kdump-config status", use_shell=False);
if len(lines) >= 1 and ": ready to kdump" in lines[0]:
use_kdump_in_cfg = read_use_kdump()
if use_kdump_in_cfg:
print('Ready')
else:
print('Not Ready')
elif not kdump_enabled:
print('Disabled')
else:
print('Ready after Reboot')

## Get the current number of kernel dump files stored
#
# @param The number of kdump files stored in /var/crash
def get_nb_dumps_in_var_crash():
(rc, lines, err_str) = run_command("find /var/crash/ -name 'kdump.*'", use_shell=False);
if rc == 0:
return len(lines)
return 0

## Command: Display kdump files
def cmd_kdump_files():
nb_dumps = get_nb_dumps_in_var_crash()
if nb_dumps == 0:
print("No kernel core dump files")
else:
(rc1, lines1, err_str) = run_command("find /var/crash/ -name 'dmesg.*'", use_shell=False);
lines1.sort(reverse=True)
(rc2, lines2, err_str) = run_command("find /var/crash/ -name 'kdump.*'", use_shell=False);
lines2.sort(reverse=True)
print("Record Key Filename")
print("-------------------------------------------------------------")
for n in range(len(lines1)):
print("%6d %s %s\n %s" % (n+1, lines1[n][11:23], lines1[n], lines2[n]))

## Command: Display kdump file (kernel log)
#
# @param num_lines Number of last lines displayed
# @param filename Name or index of the kernel log file (dmesg)
def cmd_kdump_file(num_lines, filename):
fname = None
nb_dumps = get_nb_dumps_in_var_crash()
if nb_dumps == 0:
print("Kernel crash log not found")
else:
(rc, lines, err_str) = run_command("find /var/crash/ -name 'dmesg.*'", use_shell=False);
if rc == 0 and nb_dumps == len(lines):
if filename.isdigit() and len(filename) <= 2:
num = int(filename)
if num < 1 or num > nb_dumps:
if nb_dumps == 1:
print("Invalid record number - Should be 1")
else:
print("Invalid record number - Should be between 1 and %d" % nb_dumps)
sys.exit(1)
fname = sorted(lines, reverse=True)[num-1]
else:
lines.sort(reverse=True)
for x in lines:
if x.find(filename) != -1:
fname = x
break
if fname is None:
print("Invalid key")
sys.exit(1)
(rc, lines, err_str) = run_command("/usr/bin/tail -n %d %s" % (num_lines, fname), use_shell=False);
if rc == 0:
print('File: %s' % fname)
print('\n'.join(lines))

def main():

Expand Down Expand Up @@ -714,10 +634,6 @@ def main():
parser.add_argument('--disable', action='store_true',
help='Disable kdump')

# kdump status on Current Image
parser.add_argument('--status', action='store_true',
help='Show kdump status')

# kdump status on Current Image
parser.add_argument('--status-json', action='store_true',
help='Show kdump status in json format')
Expand All @@ -726,7 +642,6 @@ def main():
parser.add_argument('--kdump-records-json', action='store_true',
help='Show kdump records in json format')

# kdump config on Current Image
# kdump config on Current Image
parser.add_argument('--config-json', action='store_true',
help='Show kdump config in json format')
Expand All @@ -739,14 +654,6 @@ def main():
parser.add_argument('--memory', nargs='?', type=str, action='store', default=False,
help='Amount of memory reserved for the capture kernel')

# Capture kernel files
parser.add_argument('--files', action='store_true',
help='Show stored capture kernel files')

# Capture kernel file
parser.add_argument('--file', nargs=1, type=str,
help='Show stored capture kernel file')

# Show more information (used for sonic-kdump-config status)
parser.add_argument("-v", "--verbose", action='store_true',
help='displays detailed kdump status information. Used with status command.')
Expand Down Expand Up @@ -776,12 +683,6 @@ def main():
cmd_kdump_memory(options.verbose, options.memory)
elif options.num_dumps != False:
cmd_kdump_num_dumps(options.verbose, options.num_dumps)
elif options.status:
cmd_kdump_status()
elif options.files != False:
cmd_kdump_files()
elif options.file:
cmd_kdump_file(options.lines, options.file[0])
elif options.dump_db:
cmd_dump_db()
elif options.status_json:
Expand Down
Loading

0 comments on commit 4cb3b72

Please sign in to comment.