| Anonymous | Login | Signup for a new account | 2009-11-22 06:42 UTC |
| Main | My View | View Issues | Roadmap | Docs |
| Viewing Issue Simple Details [ Jump to Notes ] | [ View Advanced ] [ Issue History ] [ Print ] | |||||||||||
| ID | Category | Severity | Reproducibility | Date Submitted | Last Update | |||||||
| 0003968 | [CentOS-5] kernel | crash | always | 2009-11-02 23:14 | 2009-11-18 14:52 | |||||||
| Reporter | ps7776 | View Status | public | |||||||||
| Assigned To | ||||||||||||
| Priority | normal | Resolution | open | |||||||||
| Status | new | Product Version | 5.3 | |||||||||
| Summary | 0003968: low memory exhaustion cases oom-killer to be invoked | |||||||||||
| Description |
Fairly heavily loaded server with 8 CPUs, 16 GB of RAM, 64 GB of swap running lots of memory intensive stuff plus httpd, nfs, mysql, dovecot etc. For legacy reasons we are using 32 bit PAE kernels ( which may be part of the problem ...) . System has been very stable ( normal uptime ~ 90 days ) but started crashing every few days after upgrading to 2.26.18-164 kernel. Basically what would happen is that the kernel would run fine for a few days rarely touching swap at all with LowFree hovering around 9M . However, any spike in demand for low memory driving LowFree below ~8K would start a chain reaction with oom-killer being invoked and the kernel eventually deadlocking with LowFree = 0 . The kernel was unable to free up enough low memory to bounce low memory to high memory. Increasing swappiness to close to 100 had no effect - the kernel rarely used swap. Setting overcommit_memory to 2 had no effect either and lowering lowmem_reserve_ratio ( to fex. 64 64 8 ) didn't result in any increase in LowFree Downgrading to 2.18.18-128.1.16 appears to improve the situation ( no crash yet ... ) in that LowFree is kept much, much larger ( around 320M ). So - what changed in the low memory allocation from 2.6.18-128 to 2.18.6-164 ? |
|||||||||||
| Additional Information |
|
|||||||||||
| Tags | No tags attached. | |||||||||||
| Attached Files | ||||||||||||
|
|
||||||||||||
 Relationships |
|
|
Notes |
|
|
(0010239) jhughes@hughesjr.com (administrator) 2009-11-02 23:25 |
I have had issues with VMWare host server and running out of memory, maybe try setting this variable in sysctl.conf: vm.min_free_kbytes=307200 That will try to maintain 300MB of free RAM. |
|
(0010279) ps7776 (reporter) 2009-11-04 15:37 |
After running the 2.6.18-128 PAE kernel for a while I now see that it actually behaves the same way as -164 kernel - at some point it re-balances memory such that LowFree is reduced to around 8K . However, increasing vm.min_free_kbytes from the default ~4K to 32K has a definite effect on LowFree . Maybe this isn't really a bug - it may just be a question of high demands from the hardware ( 3 gigabit ethernet cards + a fast RAID array ) capable of sustained rates of 80 Mb/s . Hmmm ... |
|
(0010280) smooge (developer) 2009-11-04 15:56 |
PAE kernels do weird things to get an architecture that is 4GB limited to work on more than that. It gets even weirder depending what the underlying CPU is (the new Nehamim allocates memory in 2GB increments per MPU which means your box works better in 6GB increments versus 4 GB like older CPUs) You might want to look at slabtop to get an idea of what is using that memory. |
|
(0010286) ps7776 (reporter) 2009-11-04 21:44 |
Right - I know that the PAE kernel is a bit of a kludge and the long term plan is to migrate everything to 64 bits. This box has two Quad-Core ( 2532 ) AMD Opterons. I'm not sure what happens when things go wrong but during "normal" running nfs4d dominates by far : 429396 429396 100% 0.34K 39036 11 156144K nfsd4_stateowners 691560 512891 74% 0.05K 9605 72 38420K buffer_head 72416 64012 88% 0.48K 9052 8 36208K ext3_inode_cache 404443 404440 99% 0.07K 7631 53 30524K nfsd4_stateids 165039 162169 98% 0.13K 5691 29 22764K dentry_cache 59402 59402 100% 0.27K 4243 14 16972K radix_tree_node 435502 435396 99% 0.03K 3854 113 15416K size-32 729 685 93% 4.00K 729 1 2916K pmd 604 589 97% 4.00K 604 1 2416K size-4096 17468 16368 93% 0.09K 397 44 1588K vm_area_struct 670 653 97% 2.00K 335 2 1340K size-2048 6240 5337 85% 0.19K 312 20 1248K filp 1568 1527 97% 0.50K 196 8 784K size-512 10089 5999 59% 0.06K 171 59 684K size-64 1683 1490 88% 0.33K 153 11 612K inode_cache 1617 1585 98% 0.35K 147 11 588K proc_inode_cache 423 420 99% 1.33K 141 3 564K task_struct 372 372 100% 1.31K 124 3 496K sighand_cache 392 392 100% 1.00K 98 4 392K size-1024 2730 2445 89% 0.12K 91 30 364K size-128 1170 1127 96% 0.25K 78 15 312K size-256 5928 5856 98% 0.05K 76 78 304K sysfs_dir_cache 73 65 89% 4.00K 73 1 292K jbd_4k 432 399 92% 0.44K 48 9 192K signal_cache 432 380 87% 0.43K 48 9 192K shmem_inode_cache 264 256 96% 0.68K 24 11 192K raid5/md0 |
|
(0010287) smooge (developer) 2009-11-04 22:17 |
Another thing that came up in LWN today was turning off the kernel overcommit. The fact that the 'vm.overcommit_memory = 2' didnt fix things kind of makes it more PAE like. Are you using NFSv4+ on the system? |
|
(0010289) ps7776 (reporter) 2009-11-04 22:41 |
Yes - see the note I added a few minutes ago. There are some twenty other boxes auto mounting directories at any given time . |
|
(0010330) ps7776 (reporter) 2009-11-10 18:01 |
I tried setting min_free_kbytes to 32768 and the 2.6.18-128 kernel ran for 7 days before going on its oom-killer spree . What puzzles me is that there isn't much difference in /proc/meminfo over the Note: 0000030 second interval that the first ( ~720 MB ) job was killed : Before : After : MemTotal: 16635504 kB MemTotal: 16635504 kB MemFree: 12062112 kB MemFree: 13484684 kB Buffers: 1900 kB Buffers: 1848 kB Cached: 2052840 kB Cached: 1340096 kB SwapCached: 0 kB SwapCached: 0 kB Active: 2533416 kB Active: 1600648 kB Inactive: 1254116 kB Inactive: 764248 kB HighTotal: 15859520 kB HighTotal: 15859520 kB HighFree: 12024792 kB HighFree: 13447952 kB LowTotal: 775984 kB LowTotal: 775984 kB LowFree: 37320 kB LowFree: 36732 kB SwapTotal: 61697624 kB SwapTotal: 61697624 kB SwapFree: 61697492 kB SwapFree: 61697492 kB Dirty: 288 kB Dirty: 252 kB Writeback: 12 kB Writeback: 0 kB AnonPages: 1732912 kB AnonPages: 1023000 kB Mapped: 72896 kB Mapped: 86956 kB Slab: 723336 kB Slab: 723472 kB PageTables: 12284 kB PageTables: 11136 kB NFS_Unstable: 0 kB NFS_Unstable: 0 kB Bounce: 0 kB Bounce: 0 kB CommitLimit: 70015376 kB CommitLimit: 70015376 kB Committed_AS: 2271788 kB Committed_AS: 1560112 kB VmallocTotal: 116728 kB VmallocTotal: 116728 kB VmallocUsed: 33656 kB VmallocUsed: 33656 kB VmallocChunk: 82492 kB VmallocChunk: 82492 kB HugePages_Total: 0 HugePages_Total: 0 HugePages_Free: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Rsvd: 0 Hugepagesize: 2048 kB Hugepagesize: 2048 kB In fact, there is no difference in low memory. Here is the output from /var/log/messages Nov 10 00:00:07 owl kernel: sshd invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0 Nov 10 00:00:07 owl kernel: [<c0457add>] out_of_memory+0x72/0x1a5 Nov 10 00:00:07 owl kernel: [<c0458fcf>] __alloc_pages+0x216/0x297 Nov 10 00:00:07 owl kernel: [<c046e31a>] cache_alloc_refill+0x26d/0x450 Nov 10 00:00:07 owl kernel: [<c046e0a3>] kmem_cache_alloc+0x41/0x4b Nov 10 00:00:07 owl kernel: [<c047e920>] getname+0x1a/0xb0 Nov 10 00:00:07 owl kernel: [<c04706ad>] do_sys_open+0x12/0xae Nov 10 00:00:07 owl kernel: [<c0470776>] sys_open+0x16/0x18 Nov 10 00:00:07 owl kernel: [<c0404ead>] sysenter_past_esp+0x56/0x79 Nov 10 00:00:07 owl kernel: ======================= Nov 10 00:00:07 owl kernel: Mem-info: Nov 10 00:00:07 owl kernel: DMA per-cpu: Nov 10 00:00:07 owl kernel: cpu 0 hot: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 0 cold: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 1 hot: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 1 cold: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 2 hot: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 2 cold: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 3 hot: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 3 cold: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 4 hot: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 4 cold: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 5 hot: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 5 cold: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 6 hot: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 6 cold: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 7 hot: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: cpu 7 cold: high 0, batch 1 used:0 Nov 10 00:00:07 owl kernel: DMA32 per-cpu: empty Nov 10 00:00:07 owl kernel: Normal per-cpu: Nov 10 00:00:07 owl kernel: cpu 0 hot: high 186, batch 31 used:68 Nov 10 00:00:07 owl kernel: cpu 0 cold: high 62, batch 15 used:58 Nov 10 00:00:07 owl kernel: cpu 1 hot: high 186, batch 31 used:31 Nov 10 00:00:08 owl kernel: cpu 1 cold: high 62, batch 15 used:56 Nov 10 00:00:08 owl kernel: cpu 2 hot: high 186, batch 31 used:30 Nov 10 00:00:08 owl kernel: cpu 2 cold: high 62, batch 15 used:53 Nov 10 00:00:08 owl kernel: cpu 3 hot: high 186, batch 31 used:29 Nov 10 00:00:08 owl kernel: cpu 3 cold: high 62, batch 15 used:59 Nov 10 00:00:08 owl kernel: cpu 4 hot: high 186, batch 31 used:53 Nov 10 00:00:08 owl kernel: cpu 4 cold: high 62, batch 15 used:60 Nov 10 00:00:08 owl kernel: cpu 5 hot: high 186, batch 31 used:28 Nov 10 00:00:08 owl kernel: cpu 5 cold: high 62, batch 15 used:47 Nov 10 00:00:08 owl kernel: cpu 6 hot: high 186, batch 31 used:174 Nov 10 00:00:08 owl kernel: cpu 6 cold: high 62, batch 15 used:48 Nov 10 00:00:08 owl kernel: cpu 7 hot: high 186, batch 31 used:21 Nov 10 00:00:08 owl kernel: cpu 7 cold: high 62, batch 15 used:50 Nov 10 00:00:08 owl kernel: HighMem per-cpu: Nov 10 00:00:08 owl kernel: cpu 0 hot: high 186, batch 31 used:111 Nov 10 00:00:08 owl kernel: cpu 0 cold: high 62, batch 15 used:8 Nov 10 00:00:08 owl kernel: cpu 1 hot: high 186, batch 31 used:180 Nov 10 00:00:08 owl kernel: cpu 1 cold: high 62, batch 15 used:8 Nov 10 00:00:08 owl kernel: cpu 2 hot: high 186, batch 31 used:27 Nov 10 00:00:08 owl kernel: cpu 2 cold: high 62, batch 15 used:10 Nov 10 00:00:08 owl kernel: cpu 3 hot: high 186, batch 31 used:26 Nov 10 00:00:08 owl kernel: cpu 3 cold: high 62, batch 15 used:14 Nov 10 00:00:08 owl kernel: cpu 4 hot: high 186, batch 31 used:162 Nov 10 00:00:08 owl kernel: cpu 4 cold: high 62, batch 15 used:9 Nov 10 00:00:08 owl kernel: cpu 5 hot: high 186, batch 31 used:11 Nov 10 00:00:08 owl kernel: cpu 5 cold: high 62, batch 15 used:7 Nov 10 00:00:08 owl kernel: cpu 6 hot: high 186, batch 31 used:133 Nov 10 00:00:08 owl kernel: cpu 6 cold: high 62, batch 15 used:8 Nov 10 00:00:08 owl kernel: cpu 7 hot: high 186, batch 31 used:31 Nov 10 00:00:08 owl kernel: cpu 7 cold: high 62, batch 15 used:0 Nov 10 00:00:08 owl kernel: Free pages: 12019504kB (11983320kB HighMem) Nov 10 00:00:08 owl kernel: Active:636501 inactive:320862 dirty:0 writeback:0 unstable:0 free:3004876 slab:180927 Nov 10 00:00:08 owl kernel: DMA free:4104kB min:584kB low:728kB high:876kB active:0kB inactive:36kB present:16384kB pages_scanned:155 all_unreclaimable? yes Nov 10 00:00:08 owl kernel: lowmem_reserve[]: 0 0 880 17008 Nov 10 00:00:08 owl kernel: DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Nov 10 00:00:08 owl kernel: lowmem_reserve[]: 0 0 880 17008 Nov 10 00:00:08 owl kernel: Normal free:32080kB min:32180kB low:40224kB high:48268kB active:1252kB inactive:1128kB present:901120kB pages_scanned:3486 all_unreclaimable? yes Nov 10 00:00:08 owl kernel: lowmem_reserve[]: 0 0 0 129024 Nov 10 00:00:08 owl kernel: HighMem free:11983320kB min:512kB low:147968kB high:295424kB active:2544756kB inactive:1282284kB present:16515072kB pages_scanned:0 all_unreclaimable? no Nov 10 00:00:08 owl kernel: lowmem_reserve[]: 0 0 0 0 Nov 10 00:00:08 owl kernel: DMA: 0*4kB 1*8kB 14*16kB 11*32kB 1*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 4104kB Nov 10 00:00:08 owl kernel: DMA32: empty Nov 10 00:00:08 owl kernel: Normal: 0*4kB 0*8kB 365*16kB 670*32kB 19*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 32080kB Nov 10 00:00:08 owl kernel: HighMem: 0*4kB 31751*8kB 33178*16kB 50726*32kB 22935*64kB 9125*128kB 9573*256kB 4843*512kB 1290*1024kB 258*2048kB 39*4096kB = 11983320kB Nov 10 00:00:08 owl kernel: 520978 pagecache pages Nov 10 00:00:08 owl kernel: Swap cache: add 51, delete 51, find 0/0, race 0+0 Nov 10 00:00:08 owl kernel: Free swap = 61697492kB Nov 10 00:00:08 owl kernel: Total swap = 61697624kB Nov 10 00:00:08 owl kernel: Free swap: 61697492kB Nov 10 00:00:08 owl kernel: 4358144 pages of RAM Nov 10 00:00:08 owl kernel: 4128768 pages of HIGHMEM Nov 10 00:00:08 owl kernel: 199268 reserved pages Nov 10 00:00:08 owl kernel: 559738 pages shared Nov 10 00:00:08 owl kernel: 0 pages swap cached Nov 10 00:00:08 owl kernel: 57 pages dirty Nov 10 00:00:08 owl kernel: 27 pages writeback Nov 10 00:00:08 owl kernel: 21378 pages mapped Nov 10 00:00:08 owl kernel: 180927 pages slab mapped-file:21378 mapped-anon:436264 pagetables:3153Nov 10 00:00:08 owl kernel: lowmem_reserve[]: 0 0 880 17008 Nov 10 00:00:08 owl kernel: DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Nov 10 00:00:08 owl kernel: lowmem_reserve[]: 0 0 880 17008 Nov 10 00:00:08 owl kernel: Normal free:32080kB min:32180kB low:40224kB high:48268kB active:1252kB inactive:1128kB prese nt:901120kB pages_scanned:3486 all_unreclaimable? yes Nov 10 00:00:08 owl kernel: lowmem_reserve[]: 0 0 0 129024 Nov 10 00:00:08 owl kernel: HighMem free:11983320kB min:512kB low:147968kB high:295424kB active:2544756kB inactive:1282284kB present:16515072kB pages_scanned:0 all_unreclaimable? no Nov 10 00:00:08 owl kernel: lowmem_reserve[]: 0 0 0 0 Nov 10 00:00:08 owl kernel: DMA: 0*4kB 1*8kB 14*16kB 11*32kB 1*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 4104kB Nov 10 00:00:08 owl kernel: DMA32: empty Nov 10 00:00:08 owl kernel: Normal: 0*4kB 0*8kB 365*16kB 670*32kB 19*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 32080kB Nov 10 00:00:08 owl kernel: HighMem: 0*4kB 31751*8kB 33178*16kB 50726*32kB 22935*64kB 9125*128kB 9573*256kB 4843*512kB 1290*1024kB 258*2048kB 39*4096kB = 11983320kB Nov 10 00:00:08 owl kernel: 520978 pagecache pages Nov 10 00:00:08 owl kernel: Swap cache: add 51, delete 51, find 0/0, race 0+0 Nov 10 00:00:08 owl kernel: Free swap = 61697492kB Nov 10 00:00:08 owl kernel: Total swap = 61697624kB Nov 10 00:00:08 owl kernel: Free swap: 61697492kB Nov 10 00:00:08 owl kernel: 4358144 pages of RAM Nov 10 00:00:08 owl kernel: 4128768 pages of HIGHMEM Nov 10 00:00:08 owl kernel: 199268 reserved pages Nov 10 00:00:08 owl kernel: 559738 pages shared Nov 10 00:00:08 owl kernel: 0 pages swap cached Nov 10 00:00:08 owl kernel: 57 pages dirty Nov 10 00:00:08 owl kernel: 27 pages writeback Nov 10 00:00:08 owl kernel: 21378 pages mapped Nov 10 00:00:08 owl kernel: 180927 pages slab Nov 10 00:00:08 owl kernel: 3153 pages pagetables Nov 10 00:00:08 owl kernel: Out of memory: Killed process 9970 (neutron_counter). I'll try increasing vm.min_free_kbytes . Anything esle I should try ? ( Apart from switching to a 64 bit kernel ) |
|
(0010353) ps7776 (reporter) 2009-11-12 16:02 |
Have tried various values of vm.min_free_kbytes but all to no avail. oom-killer eventually goes on a rampage no matter what the value is. However, monitoring /proc/slabinfo makes me suspect nfs4d : the number of state_owners and state_ids is huge when oom-killer kicks in ( around 1.5M ) . It just keeps going up and up as time goes by. This problem has actually been reported on the NFS4 mailing list ( http://linux-nfs.org/pipermail/nfsv4/2009-August/011050.html [^] ) |
|
(0010378) Zool (reporter) 2009-11-18 09:40 |
i have seen this error, too. Nov 18 00:45:59 ws-web1 kernel: httpd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Nov 18 00:45:59 ws-web1 kernel: Nov 18 00:45:59 ws-web1 kernel: Call Trace: Nov 18 00:45:59 ws-web1 kernel: [<ffffffff802beb38>] out_of_memory+0x8b/0x203 Nov 18 00:45:59 ws-web1 kernel: [<ffffffff8020f825>] __alloc_pages+0x245/0x2ce Nov 18 00:45:59 ws-web1 kernel: [<ffffffff8021377f>] __do_page_cache_readahead+0xc6/0x1ab Nov 18 00:45:59 ws-web1 kernel: [<ffffffff80214015>] filemap_nopage+0x14c/0x360 Nov 18 00:45:59 ws-web1 kernel: [<ffffffff80208ebc>] __handle_mm_fault+0x443/0x1337 Nov 18 00:45:59 ws-web1 kernel: [<ffffffff8020cb42>] _atomic_dec_and_lock+0x39/0x57 Nov 18 00:45:59 ws-web1 kernel: [<ffffffff8026766a>] do_page_fault+0xf7b/0x12e0 Nov 18 00:45:59 ws-web1 kernel: [<ffffffff8026ef1b>] monotonic_clock+0x35/0x7b Nov 18 00:45:59 ws-web1 kernel: [<ffffffff80234dc4>] do_setitimer+0x45f/0x4c7 Nov 18 00:45:59 ws-web1 kernel: [<ffffffff80264915>] _spin_lock_irq+0x9/0x14 Nov 18 00:45:59 ws-web1 kernel: [<ffffffff80229332>] do_sigaction+0x18c/0x1a0 Nov 18 00:45:59 ws-web1 kernel: [<ffffffff8026082b>] error_exit+0x0/0x6e Nov 18 00:45:59 ws-web1 kernel: Nov 18 00:45:59 ws-web1 kernel: Mem-info: Nov 18 00:45:59 ws-web1 kernel: DMA per-cpu: Nov 18 00:45:59 ws-web1 kernel: cpu 0 hot: high 186, batch 31 used:26 Nov 18 00:45:59 ws-web1 kernel: cpu 0 cold: high 62, batch 15 used:53 Nov 18 00:45:59 ws-web1 kernel: cpu 1 hot: high 186, batch 31 used:22 Nov 18 00:45:59 ws-web1 kernel: cpu 1 cold: high 62, batch 15 used:58 Nov 18 00:45:59 ws-web1 kernel: DMA32 per-cpu: empty Nov 18 00:45:59 ws-web1 kernel: Normal per-cpu: empty Nov 18 00:45:59 ws-web1 kernel: HighMem per-cpu: empty Nov 18 00:45:59 ws-web1 kernel: Free pages: 9024kB (0kB HighMem) Nov 18 00:45:59 ws-web1 kernel: Active:650516 inactive:598639 dirty:0 writeback:0 unstable:0 free:2256 slab:8569 mapped-file:1 mapped-anon:485589 pagetables:13805 Nov 18 00:45:59 ws-web1 kernel: DMA free:9024kB min:9156kB low:11444kB high:13732kB active:2601940kB inactive:2394556kB present:5242880kB pages_scanned:20462072 all_unreclaimable? yes Nov 18 00:45:59 ws-web1 kernel: lowmem_reserve[]: 0 0 0 0 Nov 18 00:45:59 ws-web1 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Nov 18 00:45:59 ws-web1 kernel: lowmem_reserve[]: 0 0 0 0 Nov 18 00:45:59 ws-web1 kernel: Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Nov 18 00:45:59 ws-web1 kernel: lowmem_reserve[]: 0 0 0 0 Nov 18 00:45:59 ws-web1 kernel: HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Nov 18 00:45:59 ws-web1 kernel: lowmem_reserve[]: 0 0 0 0 Nov 18 00:45:59 ws-web1 kernel: DMA: 4*4kB 0*8kB 3*16kB 6*32kB 11*64kB 3*128kB 2*256kB 2*512kB 2*1024kB 0*2048kB 1*4096kB = 9024kB Nov 18 00:45:59 ws-web1 kernel: DMA32: empty Nov 18 00:45:59 ws-web1 kernel: Normal: empty Nov 18 00:45:59 ws-web1 kernel: HighMem: empty Nov 18 00:45:59 ws-web1 kernel: 764206 pagecache pages Nov 18 00:45:59 ws-web1 kernel: Swap cache: add 7352701, delete 7352708, find 2397646/3001624, race 0+53 Nov 18 00:45:59 ws-web1 kernel: Free swap = 0kB Nov 18 00:45:59 ws-web1 kernel: Total swap = 3149392kB Nov 18 00:45:59 ws-web1 kernel: Free swap: 0kB Nov 18 00:45:59 ws-web1 kernel: 1310720 pages of RAM Nov 18 00:45:59 ws-web1 kernel: 29034 reserved pages Nov 18 00:45:59 ws-web1 kernel: 104002 pages shared Nov 18 00:45:59 ws-web1 kernel: 1 pages swap cached Nov 18 02:41:41 ws-web1 kernel: httpd[20949]: segfault at 00007fffb8f8bf80 rip 00002abf1e781b0d rsp 00007fffb8f8bf60 error 6 Linux ws-web1 2.6.18-164.el5xen 0000001 SMP Thu Sep 3 04:03:03 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux [root@ws-web1 httpd]# free -m total used free shared buffers cached Mem: 5120 5106 13 0 64 2173 -/+ buffers/cache: 2869 2250 Swap: 3075 451 2623 |
|
(0010380) ps7776 (reporter) 2009-11-18 14:52 |
I believe this issue has been fixed in kernel 2.6.18-164.6.1 . At least there are comments in the release notes to that effect . So I installed this kernel even though it is not part of 5.3 ( it is in 5.4 updates only ) . I didn't install any other packages from 5.4 . Has been running without any problems for a week now. The number of NFS stateowners is reasonable and the memory used is believable. Whether or not there still is a very small memory leak is hard to tell. I think the number more or less scales with the number of cached inodes which sort of sounds reasonable. |
|
Issue History |
|||
| Date Modified | Username | Field | Change |
| 2009-11-02 23:14 | ps7776 | New Issue | |
| 2009-11-02 23:14 | ps7776 | Assigned To | => kbsingh@karan.org |
| 2009-11-02 23:25 | jhughes@hughesjr.com | Note Added: 0010239 | |
| 2009-11-04 15:37 | ps7776 | Note Added: 0010279 | |
| 2009-11-04 15:56 | smooge | Note Added: 0010280 | |
| 2009-11-04 21:44 | ps7776 | Note Added: 0010286 | |
| 2009-11-04 22:17 | smooge | Note Added: 0010287 | |
| 2009-11-04 22:41 | ps7776 | Note Added: 0010289 | |
| 2009-11-10 18:01 | ps7776 | Note Added: 0010330 | |
| 2009-11-12 16:02 | ps7776 | Note Added: 0010353 | |
| 2009-11-18 09:40 | Zool | Note Added: 0010378 | |
| 2009-11-18 09:50 | Zool | Issue Monitored: Zool | |
| 2009-11-18 14:52 | ps7776 | Note Added: 0010380 | |
| Copyright © 2000 - 2009 Mantis Group |  |
