Re: random oops crashes

[Vasu Vuppala]

Forgot to mention (since it is occuring in the networking code),
the two machines we tested had different NIC cards (8139too,
etherexpress pro); so it may not be the drivers either.

Thanks.

- vasu

--- Vasu Vuppala <[e-mail]> wrote:
> Hi,
> 
> Our firewall has been crashing a lot recently; it had
> been working fine for months. We replaced it with another
> machine but the problem remains. So it is definitely not a hardware
> issue. The crashes are very random. The machine would run
> for weeks without any crash but then will crash 10-15
> times in a day. 
> 
> ksymoops indicates that the crash always occurs at
> the same function (_free_pages) and instruction 
> (lock decl 0x14(%ecx)). ksymoops output is attached.
> 
> The machine has minimal software on it. It only runs:
> sshd, crond, klogd, mingetty, syslogd, apart from kernel 
> daemons and init. It uses iptables for NAT.
> 
> Info about the machine:
>   Distribution: Red Hat 7.1
>   Kernel: 2.4.2-2smp
> 
>  The two PCs that were used: Dell Dimension L667r, and DigiLink.
>   Processor: Pentium III
>   Memory: 128MB
> 
> Can anyone tell us what the problem is.
> Plese let me know if you need more information.
> 
> Thank you.
>   
> - vasu
> 
> __________________________________________________
> Do You Yahoo!?
> Send FREE video emails in Yahoo! Mail!
> http://promo.yahoo.com/videomail/> ksymoops 2.4.0 on i686
2.4.2-2smp.  Options used
>      -V (default)
>      -k /proc/ksyms (default)
>      -l /proc/modules (default)
>      -o /lib/modules/2.4.2-2smp/ (default)
>      -m /boot/System.map-2.4.2-2smp (default)
> 
> Warning: You did not tell me where to find symbol information.  I
> will
> assume that the log matches the kernel and modules that are running
> right now and I'll use the default options above for symbol
> resolution.
> If the current kernel and/or modules do not match the log, you can
> get
> more accurate output by telling me the kernel version and where to
> find
> map, modules, ksyms etc.  ksymoops -h explains the options.
> 
> Warning (compare_maps): ksyms_base symbol
> __VERSIONED_SYMBOL(shmem_file_setup) not found in System.map. 
> Ignoring ksyms_base entry
> Warning (compare_maps): mismatch on symbol partition_name  ,
> ksyms_base says c01bba60, System.map says c015d2a0.  Ignoring
> ksyms_base entry
> Warning (compare_maps): mismatch on symbol usb_devfs_handle  ,
> usbcore says c882ec60,
> /lib/modules/2.4.2-2smp/kernel/drivers/usb/usbcore.o says c882e780.
>  Ignoring /lib/modules/2.4.2-2smp/kernel/drivers/usb/usbcore.o
> entry
> Unable to handle kernel paging request at virtual address d70f589b
> c0133c32
> Oops: 0002
> CPU:    0
> EIP:    0010:[<c0133c32>]
> Using defaults from ksymoops -t elf32-i386 -a i386
> EFLAGS: 00010202
> eax: d70f5887   ebx: 00000001   ecx: d70f5887   edx: 00000000
> esi: c1397bc0   edi: c4d36980   ebp: c4d36880   esp: c0293ccc
> ds: 0018   es: 0018   ss: 0018
> Process swapper (pid: 0, stackpage=c0293000)
> Stack: c01c49fe c4d36980 c1397bc0 c01c5025 c1397bc0 00000060
> c1397bc0 c425fc00
>        00000000 c7dffc20 c01c8738 c1397bc0 00000002 0000003c
> c1397bc0 c7dffc20
>        c01cc696 c0292000 0000003c c1397bc0 c7dffc20 c01cc716
> c1397bc0 c425fc00
> Call Trace: [<c01c49fe>] [<c01c5025>] [<c01c8738>] [<c01cc696>]
> [<c01cc716>] [<c
> 01dce20>] [<c01cea2e>]
>        [<c01dcecb>] [<c01dce20>] [<c01ced33>] [<c01dce20>]
> [<c01ced6a>] [<c01da3
>        [<c8845740>] [<c01da427>] [<c01ced6a>] [<c01da348>]
> [<c01da3b0>] [<c01d94
>        [<c01d9350>] [<c01ced6a>] [<c01f62f4>] [<c01c8a2c>]
> [<c01d9196>] [<c01d93
>        [<c01c8ebb>] [<c010aa1e>] [<c011d51b>] [<c010ac65>]
> [<c0107290>] [<c01072
>        [<c0107290>] [<c01072bc>] [<c0107342>] [<c0105000>]
> [<c01001cf>]
> Code: f0 ff 49 14 0f 94 c0 84 c0 74 07 89 c8 e9 9c f5 ff ff c3 8d
> 
> >>EIP; c0133c32 <__free_pages+2/20>   <=====
> Trace; c01c49fe <skb_release_data+4e/80>
> Trace; c01c5025 <skb_linearize+c5/130>
> Trace; c01c8738 <dev_queue_xmit+a8/2b0>
> Trace; c01cc696 <neigh_resolve_output+f6/1f0>
> Trace; c01cc716 <neigh_resolve_output+176/1f0>
> Code;  c0133c32 <__free_pages+2/20>
> 00000000 <_EIP>:
> Code;  c0133c32 <__free_pages+2/20>   <=====
>    0:   f0 ff 49 14               lock decl 0x14(%ecx)   <=====
> Code;  c0133c36 <__free_pages+6/20>
>    4:   0f 94 c0                  sete   %al
> Code;  c0133c39 <__free_pages+9/20>
>    7:   84 c0                     test   %al,%al
> Code;  c0133c3b <__free_pages+b/20>
>    9:   74 07                     je     12 <_EIP+0x12> c0133c44
> <__free_pages+14/20>
> Code;  c0133c3d <__free_pages+d/20>
>    b:   89 c8                     mov    %ecx,%eax
> Code;  c0133c3f <__free_pages+f/20>
>    d:   e9 9c f5 ff ff            jmp    fffff5ae <_EIP+0xfffff5ae>
> c01331e0 <__free_pages_ok+0/380>
> Code;  c0133c44 <__free_pages+14/20>
>   12:   c3                        ret    
> Code;  c0133c45 <__free_pages+15/20>
>   13:   8d 00                     lea    (%eax),%eax
> 
> Kernel panic: Aiee, killing interrupt handler!
> 
> 4 warnings issued.  Results may not be reliable.
> 

__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/