User:Trogod/System0KernCrash1

This morning, my User:Trogod/System0 was unresponsive! It appears to have been the victim of a crash. https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies

Filed as https://bugs.launchpad.net/ubuntu/+source/linux/+bug/404803

$ uname -a
Linux foofoo 2.6.28-13-generic #45-Ubuntu SMP Tue Jun 30 22:12:12 UTC 2009 x86_64 GNU/Linux
$ cat /proc/version_signature
Ubuntu 2.6.28-13.45-generic

from my kern.log

0fe000+20000]
Jul 24 14:38:06 foofoo kernel: [1437044.520498] BUG: soft lockup - CPU#1 stuck for 61s! [mythfrontend.re:31369]
Jul 24 14:38:06 foofoo kernel: [1437044.520501] Modules linked in: bridge stp bnep video output input_polldev jfs lp snd_hda_intel snd_pc
m_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device nvidia(P
) psmouse ppdev snd soundcore serio_raw pcspkr shpchp snd_page_alloc parport_pc parport joydev usbhid ohci1394 ieee1394 forcedeth fbcon til
eblit font bitblit softcursor
Jul 24 14:38:06 foofoo kernel: [1437044.520501] CPU 1:
Jul 24 14:38:06 foofoo kernel: [1437044.520501] Modules linked in: bridge stp bnep video output input_polldev jfs lp snd_hda_intel snd_pc
m_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device nvidia(P
) psmouse ppdev snd soundcore serio_raw pcspkr shpchp snd_page_alloc parport_pc parport joydev usbhid ohci1394 ieee1394 forcedeth fbcon til
eblit font bitblit softcursor
Jul 24 14:38:06 foofoo kernel: [1437044.520501] Pid: 31369, comm: mythfrontend.re Tainted: P           2.6.28-13-generic #45-Ubuntu
Jul 24 14:38:06 foofoo kernel: [1437044.520501] RIP: 0010:[<ffffffffa04639a5>]  [<ffffffffa04639a5>] _nv004776rm+0x13/0x14c [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501] RSP: 0018:ffff88006ada9a88  EFLAGS: 00200297
Jul 24 14:38:06 foofoo kernel: [1437044.520501] RAX: ffff88006dc0b000 RBX: ffff880022b85cd8 RCX: 0000000000000000
Jul 24 14:38:06 foofoo kernel: [1437044.520501] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff88006dc0b000
Jul 24 14:38:06 foofoo kernel: [1437044.520501] RBP: ffff880022b85cd8 R08: ffff88006dc0b000 R09: 0000000000000018
Jul 24 14:38:06 foofoo kernel: [1437044.520501] R10: ffff880022b85d10 R11: 0000000000200202 R12: ffff88006ada9a78
Jul 24 14:38:06 foofoo kernel: [1437044.520501] R13: ffff88006ada9a78 R14: ffff880022b85d10 R15: 0000000000200202
Jul 24 14:38:06 foofoo kernel: [1437044.520501] FS:  00007f7899697950(0000) GS:ffff88006f802b80(0000) knlGS:0000000000000000
Jul 24 14:38:06 foofoo kernel: [1437044.520501] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Jul 24 14:38:06 foofoo kernel: [1437044.520501] CR2: 00007f78bf595a90 CR3: 000000006ec90000 CR4: 00000000000406a0
Jul 24 14:38:06 foofoo kernel: [1437044.520501] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul 24 14:38:06 foofoo kernel: [1437044.520501] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul 24 14:38:06 foofoo kernel: [1437044.520501] Call Trace:
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa032ca9e>] ? _nv009808rm+0x1f/0x4f [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa032fbe2>] ? _nv009755rm+0x39/0x7a [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa043d035>] ? _nv009841rm+0x9/0x28 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa043d00a>] ? _nv009843rm+0x2d/0x4f [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa043ca69>] ? _nv009488rm+0x1c/0x15a [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa050d58a>] ? _nv001686rm+0x63/0x11f [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa050ca75>] ? _nv001676rm+0x1f/0x2c [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa050d7ce>] ? _nv001738rm+0xa3/0xf0 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa050bdf8>] ? _nv001757rm+0xe1/0x122 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa0440321>] ? _nv010151rm+0xc4/0x1a6 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa0132c4b>] ? _nv017637rm+0xf4/0x211 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa0132f5a>] ? _nv017639rm+0x1f2/0x206 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa027f351>] ? _nv012700rm+0x42f/0x49b [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa027c5be>] ? _nv012901rm+0xbe2/0xc88 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa03ee7e6>] ? _nv003184rm+0x64a/0x8a8 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa03ee77a>] ? _nv003184rm+0x5de/0x8a8 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa04139d5>] ? _nv006002rm+0x96/0xd4 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa0293375>] ? _nv003189rm+0x455/0x6fd [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa049245d>] ? rm_ioctl+0x2f/0x67 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa054fdb3>] ? nv_kern_ioctl+0x1b3/0x480 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffffa05500bc>] ? nv_kern_unlocked_ioctl+0x1c/0x20 [nvidia]
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffff802f62a1>] ? vfs_ioctl+0x31/0xa0
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffff802f6655>] ? do_vfs_ioctl+0x75/0x230
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffff802f68a9>] ? sys_ioctl+0x99/0xa0
Jul 24 14:38:06 foofoo kernel: [1437044.520501]  [<ffffffff8021253a>] ? system_call_fastpath+0x16/0x1b
Jul 24 14:39:11 foofoo kernel: [1437110.016497] BUG: soft lockup - CPU#1 stuck for 61s! [mythfrontend.re:31369]
Jul 24 14:39:11 foofoo kernel: [1437110.016501] Modules linked in: bridge stp bnep video output input_polldev jfs lp snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device nvidia(P) psmouse ppdev snd soundcore serio_raw pcspkr shpchp snd_page_alloc parport_pc parport joydev usbhid ohci1394 ieee1394 forcedeth fbcon tileblit font bitblit softcursor

and it repeats similar messages over and over until finally

Jul 25 06:10:46 foofoo kernel: [1492905.783161]  [<ffffffff8021253a>] ? system_call_fastpath+0x16/0x1b
Jul 25 06:11:11 foofoo kernel: [1492905.783166]  [<ffffffff80210373>] sys_clone+0x23/0x30
Jul 25 06:11:22 foofoo kernel: [1492905.783169]  [<ffffffff802129c7>] ptregscall_common+0x67/0xb0
Jul 25 06:11:45 foofoo kernel: [1492905.783171] Mem-Info:
Jul 25 06:12:06 foofoo kernel: [1492905.783173] DMA per-cpu:
Jul 25 06:12:18 foofoo kernel: [1492905.783175] CPU    0: hi:    0, btch:   1 usd:   0
Jul 25 06:12:31 foofoo kernel: [1492905.783177] CPU    1: hi:    0, btch:   1 usd:   0
Jul 25 06:12:40 foofoo kernel: [1492905.783179] DMA32 per-cpu:
Jul 25 06:12:51 foofoo kernel: [1492905.783181] CPU    0: hi:  186, btch:  31 usd:   0
Jul 25 06:13:08 foofoo kernel: [1492905.783183] CPU    1: hi:  186, btch:  31 usd:   0
Jul 25 06:13:23 foofoo kernel: [1492905.783186] Active_anon:33 active_file:31 inactive_anon:31
Jul 25 06:13:40 foofoo kernel: [1492905.783188]  inactive_file:48 unevictable:0 dirty:0 writeback:28 unstable:0
Jul 25 06:13:58 foofoo kernel: [1492905.783189]  free:5774 slab:376309 mapped:2088 pagetables:2730 bounce:0
Jul 25 06:14:01 foofoo kernel: [1492905.783193] DMA free:32kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0
kB inactive_file:0kB unevictable:0kB present:5540kB pages_scanned:0 all_unreclaimable? no
Jul 25 06:14:01 foofoo kernel: [1492905.783196] lowmem_reserve[]: 0 1750 1750 1750
Jul 25 06:14:01 foofoo kernel: [1492905.783202] DMA32 free:23064kB min:5340kB low:6672kB high:8008kB active_anon:132kB inactive_anon:124k
B active_file:124kB inactive_file:192kB unevictable:0kB present:1792684kB pages_scanned:0 all_unreclaimable? no
Jul 25 06:14:01 foofoo kernel: [1492905.783206] lowmem_reserve[]: 0 0 0 0
Jul 25 06:14:01 foofoo kernel: [1492905.783210] DMA: 2*4kB 3*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 
= 32kB
Jul 25 06:14:01 foofoo kernel: [1492905.783219] DMA32: 5785*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*40
96kB = 23140kB
Jul 25 06:14:01 foofoo kernel: [1492905.783229] 142 total pagecache pages
Jul 25 06:14:01 foofoo kernel: [1492905.783230] 64 pages in swap cache
Jul 25 06:14:01 foofoo kernel: [1492905.783233] Swap cache stats: add 6398638, delete 6398574, find 11595387/12800725
Jul 25 06:14:01 foofoo kernel: [1492905.783235] Free swap  = 9509328kB
Jul 25 06:14:01 foofoo kernel: [1492905.783237] Total swap = 10000452kB
Jul 25 06:14:05 foofoo kernel: [1492905.793898] 458480 pages RAM
Jul 25 06:14:22 foofoo kernel: [1492905.793901] 14429 pages reserved
Jul 25 06:14:46 foofoo kernel: [1492905.793903] 2725 pages shared
Jul 25 06:15:06 foofoo kernel: [1492905.793905] 413139 pages non-shared
Jul 25 06:15:35 foofoo kernel: [1492905.793909] Out of memory: kill process 3677 (xfce4-session) score 15027 or a child
Jul 25 06:16:05 foofoo kernel: [1492905.793936] Killed process 3782 (xfce4-panel)
Jul 25 06:16:10 foofoo kernel: [1492906.396071] cron invoked oom-killer: gfp_mask=0x40d0, order=1, oomkilladj=0
Jul 25 06:16:24 foofoo kernel: [1492906.396074] Pid: 3387, comm: cron Tainted: P           2.6.28-13-generic #45-Ubuntu

that's the last line until my reboot at about 9:30am

Jul 25 09:35:07 foofoo kernel: Inspecting /boot/System.map-2.6.28-13-generic
Jul 25 09:35:07 foofoo kernel: Cannot find map file.
Jul 25 09:35:07 foofoo kernel: Loaded 74991 symbols from 39 modules.
Jul 25 09:35:07 foofoo kernel: [    0.000000] BIOS EBDA/lowmem at: 0009e800/0009e800
Jul 25 09:35:07 foofoo kernel: [    0.000000] Initializing cgroup subsys cpuset
Jul 25 09:35:07 foofoo kernel: [    0.000000] Initializing cgroup subsys cpu
Jul 25 09:35:07 foofoo kernel: [    0.000000] Linux version 2.6.28-13-generic (buildd@yellow) (gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4) ) #45-Ubuntu SMP Tue Jun 30 22:12:12 UTC 2009 (Ubuntu 2.6.28-13.45-generic)

I made a directory "kernbug" on my machine and put the following in it

$ mkdir ~/kernbug
$ mkdir ~/kernbug/1
$ dmesg > ~/kernbug/1/dmesg.log
$ sudo lspci -vvnn > ~/kernbug/1/lspci-vvnn.log
[sudo] password for foo: 
$ ls -Ftl ~/kernbug/1/
total 68
-rw-r--r-- 1 foo foo 16482 2009-07-26 00:39 lspci-vvnn.log
-rw-r--r-- 1 foo foo 46057 2009-07-26 00:38 dmesg.log
$ cp kern.log kern.log.0 ~/kernbug/1/.
$ cp dmesg dmesg.0 ~/kernbug/1/.

Ahhh what the hell, I went ahead and filed a kernel bug using

ubuntu-bug -p linux

$ ubuntu-bug -p linux

*** Collecting problem information

The collected information can be sent to the developers to improve the
application. This might take a few minutes.
......

*** Send problem report to the developers?

After the problem report has been sent, please fill out the form in the
automatically opened web browser.

What would you like to do? Your options are:
  S: Send report (65.9 KiB)
  V: View report
  K: Keep report file for sending later or copying to somewhere else
  C: Cancel
Please choose (S/V/K/C): S

*** Uploading problem information

The collected information is being sent to the bug tracking system.
This might take a few minutes.

after that, a web browser appeared, I made an account, filed the bug and we'll see what happens