kernel panic with Hardware flow offloading on ubnt-x (MT7621)
Username: mistepien
Origin: https://bugs.openwrt.org/index.php?do=details&task_id=2115
hardware ubnt-x (edgerouter-x)
openwrt: OpenWrt SNAPSHOT, r9293-e3311cb1
I turn on hardware offload and system went south (hangup). Automatic hard
reset (by kernel itself or because of watchcat)
But syslogd logs to other machine and
this is what I found in logs:
Feb 10 21:04:59 thorgal kernel: [ 6300.830740] (t=6003 jiffies g=60795 c=60794 q=1375) Feb 10 21:04:59 thorgal kernel: [ 6300.860492] (detected by 1, t=6006 jiffies, g=60795, c=60794, q=1375) Feb 10 21:04:59 thorgal kernel: [ 6300.863610] rcu_sched I Feb 10 21:04:59 thorgal kernel: [ 6300.884363] Sending NMI from CPU 1 to CPUs 3: Feb 10 21:04:59 thorgal kernel: [ 6300.897157] 0 8 2 0x00100000 Feb 10 21:04:59 thorgal kernel: [ 6300.919637] Stack : 805b0000 8fc42bc0 8fc5fe30 8fc5fe30 81222380 8fc5fe30 805b0000 81222380 Feb 10 21:04:59 thorgal kernel: [ 6300.936268] 00000003 80086ddc 000912c6 81222380 81222380 8fc5fe30 805b0000 81222380 Feb 10 21:04:59 thorgal kernel: [ 6300.952899] 00000003 805ae1b8 805b6550 80491eec 00000001 8fc5fe30 805b0000 000912c6 Feb 10 21:04:59 thorgal kernel: [ 6300.969530] 81222380 80494e14 00000003 80082e28 805b0000 00000000 8fc5febc 00000001 Feb 10 21:04:59 thorgal kernel: [ 6300.986160] 00000000 812223f4 000912c6 8008732c 8fc42bc0 01c00001 00000001 805b6440 Feb 10 21:04:59 thorgal kernel: [ 6301.002793] ... Feb 10 21:04:59 thorgal kernel: [ 6301.007654] Call Trace: Feb 10 21:04:59 thorgal kernel: [ 6301.012531] [<80491dcc>] __schedule+0x7ac/0x860 Feb 10 21:04:59 thorgal kernel: [ 6301.021543] [<80491eec>] schedule+0x6c/0x84 Feb 10 21:04:59 thorgal kernel: [ 6301.029863] [<80494e14>] schedule_timeout+0x17c/0x1b8 Feb 10 21:04:59 thorgal kernel: [ 6301.039912] [<80083444>] rcu_gp_kthread+0x560/0x950 Feb 10 21:04:59 thorgal kernel: [ 6301.049626] [<8004c1b8>] kthread+0x130/0x144 Feb 10 21:04:59 thorgal kernel: [ 6301.058120] [<8000b0d8>] ret_from_kernel_thread+0x14/0x1c Feb 10 21:04:59 thorgal kernel: [ 6304.380977] NMI backtrace for cpu 3 Feb 10 21:04:59 thorgal kernel: [ 6304.387914] CPU: 3 PID: 2514 Comm: kworker/3:1 Not tainted 4.14.98 #0 Feb 10 21:04:59 thorgal kernel: [ 6304.400742] Workqueue: events rht_deferred_worker Feb 10 21:04:59 thorgal kernel: [ 6304.410088] Stack : 8053ac44 8fc15d24 805b0000 8e9f3280 8053acd8 00000003 805b6440 805ae0ac Feb 10 21:04:59 thorgal kernel: [ 6304.426716] 000000e0 800705fc 8ff0af3c 805b0947 805b0000 00000001 8fc15ce8 53261683 Feb 10 21:04:59 thorgal kernel: [ 6304.443343] 00000000 00000000 00000000 00003c40 00000000 000000dd 00000008 00000000 Feb 10 21:04:59 thorgal kernel: [ 6304.459969] 00000000 805b0000 00061d66 00000000 00000000 00000000 805d0000 00000003 Feb 10 21:04:59 thorgal kernel: [ 6304.476595] 805b6440 805ae0ac 000000e0 805b0000 00000000 802a4378 0000000c 8061000c Feb 10 21:04:59 thorgal kernel: [ 6304.493222] ... Feb 10 21:04:59 thorgal kernel: [ 6304.498076] Call Trace: Feb 10 21:04:59 thorgal kernel: [ 6304.502943] [<800106c0>] show_stack+0x58/0x100 Feb 10 21:04:59 thorgal kernel: [ 6304.511777] [<80478bfc>] dump_stack+0x9c/0xe0 Feb 10 21:04:59 thorgal kernel: [ 6304.520447] [<8047fa94>] nmi_cpu_backtrace+0xe4/0x108 Feb 10 21:04:59 thorgal kernel: [ 6304.530488] [<8047fb80>] nmi_trigger_cpumask_backtrace+0xc8/0x168 Feb 10 21:04:59 thorgal kernel: [ 6304.542606] [<80084dd8>] rcu_dump_cpu_stacks+0xc4/0x13c Feb 10 21:04:59 thorgal kernel: [ 6304.552993] [<80084228>] rcu_check_callbacks+0x3b4/0x8b8 Feb 10 21:04:59 thorgal kernel: [ 6304.563560] [<800877a8>] update_process_times+0x34/0x7c Feb 10 21:04:59 thorgal kernel: [ 6304.573963] [<8009838c>] tick_handle_periodic+0x34/0xd0 Feb 10 21:04:59 thorgal kernel: [ 6304.584364] [<803275e0>] gic_compare_interrupt+0x70/0x80 Feb 10 21:04:59 thorgal kernel: [ 6304.594930] [<80077ba0>] handle_percpu_devid_irq+0xbc/0x1a8 Feb 10 21:04:59 thorgal kernel: [ 6304.606024] [<80071ef0>] generic_handle_irq+0x40/0x58 Feb 10 21:04:59 thorgal kernel: [ 6304.616071] [<8025339c>] gic_handle_local_int+0x9c/0xe8 Feb 10 21:04:59 thorgal kernel: [ 6304.626458] [<8025355c>] gic_irq_dispatch+0x10/0x20 Feb 10 21:04:59 thorgal kernel: [ 6304.636153] [<80071ef0>] generic_handle_irq+0x40/0x58 Feb 10 21:04:59 thorgal kernel: [ 6304.646199] [<80495fcc>] do_IRQ+0x1c/0x2c Feb 10 21:04:59 thorgal kernel: [ 6304.654166] [<8025252c>] plat_irq_dispatch+0xfc/0x138 Feb 10 21:04:59 thorgal kernel: [ 6304.664209] [<8000b5e8>] except_vec_vi_end+0xb8/0xc4 Feb 10 21:04:59 thorgal kernel: [ 6304.674076] [<8006a94c>] queued_spin_lock_slowpath+0x1dc/0x2e0 Feb 10 21:04:59 thorgal kernel: [ 6304.685677] [<802386ac>] rht_deferred_worker+0x20c/0x4f0 Feb 10 21:04:59 thorgal kernel: [ 6304.696236] [<80046230>] process_one_work+0x258/0x400 Feb 10 21:04:59 thorgal kernel: [ 6304.706276] [<80046728>] worker_thread+0x350/0x5b4 Feb 10 21:04:59 thorgal kernel: [ 6304.715806] [<8004c1b8>] kthread+0x130/0x144 Feb 10 21:04:59 thorgal kernel: [ 6304.724291] [<8000b0d8>] ret_from_kernel_thread+0x14/0x1c