VDS error

[fandikurnia]

Hi,

I'm get some strange message log on kernel. In this situation, I cannot ssh from remote app (putty), and i try direct from grid controller, using ca <name_app>.
After restart vds, now it's working again. For log information :

Sep 29 10:02:56 cpanelcloud kernel: ffffff8025a5e2>] __do_page_cache_readahead+0x152/0x330
Sep 29 10:03:18 cpanelcloud kernel: [<ffffffff803a6968>] io_schedule+0x28/0x40
Sep 29 10:03:28 cpanelcloud kernel: [<ffffffff803a6ce8>] __wait_on_bit_lock+0x68/0x80
Sep 29 10:03:39 cpanelcloud kernel: [<ffffffff80253430>] sync_page+0x0/0x50
Sep 29 10:03:46 cpanelcloud kernel: [<ffffffff8025ab91>] do_page_cache_readahead+0x61/0x80
Sep 29 10:03:48 cpanelcloud kernel: [<ffffffff802563a0>] filemap_nopage+0x150/0x350
Sep 29 10:03:49 cpanelcloud kernel: [<ffffffff80263570>] __handle_mm_fault+0x750/0x1210
Sep 29 10:03:49 cpanelcloud kernel: [<ffffffff8028a229>] path_release+0x29/0x30
Sep 29 10:03:49 cpanelcloud kernel: [<ffffffff80300eeb>] __up_read+0x9b/0xb0
Sep 29 10:03:49 cpanelcloud kernel: [<ffffffff80215ac9>] do_page_fault+0xcc9/0x1140
Sep 29 10:03:49 cpanelcloud kernel: [<ffffffff802099c7>] error_exit+0x0/0x71
Sep 29 10:03:49 cpanelcloud kernel:
Sep 29 10:03:49 cpanelcloud kernel: Mem-info:
Sep 29 10:03:49 cpanelcloud kernel: DMA per-cpu:
Sep 29 10:03:49 cpanelcloud kernel: cpu 0 hot: high 0, batch 1 used:0
Sep 29 10:03:49 cpanelcloud kernel: cpu 0 cold: high 0, batch 1 used:0
Sep 29 10:03:49 cpanelcloud kernel: DMA32 per-cpu:
Sep 29 10:03:49 cpanelcloud kernel: cpu 0 hot: high 186, batch 31 used:152
Sep 29 10:03:49 cpanelcloud kernel: cpu 0 cold: high 62, batch 15 used:14
Sep 29 10:03:49 cpanelcloud kernel: Normal per-cpu: empty
Sep 29 10:03:49 cpanelcloud kernel: HighMem per-cpu: empty
Sep 29 10:03:49 cpanelcloud kernel: Free pages: 8396kB (0kB HighMem)
Sep 29 10:03:49 cpanelcloud kernel: Active:232197 inactive:1349 dirty:0 writeback:0 unstable:0 free:2099 slab:5364 mapped:814
pagetables:10771
Sep 29 10:03:49 cpanelcloud kernel: DMA free:4036kB min:28kB low:32kB high:40kB active:6100kB inactive:0kB present:7536kB page
s_scanned:11181 all_unreclaimable? yes
Sep 29 10:03:49 cpanelcloud kernel: lowmem_reserve[]: 0 1002 1002 1002
Sep 29 10:03:49 cpanelcloud kernel: DMA32 free:4360kB min:4032kB low:5040kB high:6048kB active:922688kB inactive:5396kB presen
t:1026160kB pages_scanned:2043805 all_unreclaimable? yes
Sep 29 10:03:49 cpanelcloud kernel: lowmem_reserve[]: 0 0 0 0
Sep 29 10:03:49 cpanelcloud kernel: Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned
:0 all_unreclaimable? no
Sep 29 10:03:49 cpanelcloud kernel: lowmem_reserve[]: 0 0 0 0
Sep 29 10:03:49 cpanelcloud kernel: HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_
scanned:0 all_unreclaimable? no
Sep 29 10:03:49 cpanelcloud kernel: lowmem_reserve[]: 0 0 0 0
Sep 29 10:03:49 cpanelcloud kernel: DMA: 3*4kB 1*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB =
Sep 29 10:03:49 cpanelcloud kernel: Normal: empty
Sep 29 10:03:49 cpanelcloud kernel: HighMem: empty
Sep 29 10:03:49 cpanelcloud kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Sep 29 10:03:49 cpanelcloud kernel: Free swap = 0kB
Sep 29 10:03:49 cpanelcloud kernel: Total swap = 0kB
Sep 29 10:03:49 cpanelcloud kernel: Free swap: 0kB
Sep 29 10:03:49 cpanelcloud kernel: 264192 pages of RAM
Sep 29 10:03:49 cpanelcloud kernel: 5882 reserved pages
Sep 29 10:03:49 cpanelcloud kernel: 125608 pages shared
Sep 29 10:03:49 cpanelcloud kernel: 0 pages swap cached
Sep 29 10:03:49 cpanelcloud kernel: httpd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Sep 29 10:03:49 cpanelcloud kernel:


I hope someone can explain, what the trigger for this error?
Thank's


Warm Regards

Fandi kurnia

[fandikurnia]

I found solution from Centos forum :

A possible answer:

Apache should not eat so much. Add to httpd.conf line "RLimitMEM
67108864. That should limit each process to 64MB of memory.
As a general idea, you managed to figure out when you get to eat so
much? It would be better if you stay with "top" while increasing the
memory consumption RES column to see who consume the most.

My answer:
I think it's very right about the memory limitation of httpd processes
(but I think I will greatly aggravate the web, using a lot until it
connects mysql .......), but try this one next week not a negativist try
to answer.
How about the top not really the server freezes randomly (and not find
anything in logs except the out of memory OOM killed httpd) and what is
more curious is that it freeze at night when traffic isn't so much.
I took more into account but in my opinion (a kernel bug) and those
above and on other forums from abroad (because of mysql) have left only
these two possible problems. Meanwhile I read and took a crash solution
from centos.org forum that to include a cron with a httpd restart and
then swapoff -a and swapon -a at every 6 hours and hope has to issue you
a swap but ........ ... We look forward ideas or answers.
Thanks for your intention to help!!!!!!

https://www.centos.org/modules/newbb/viewtopic.php?topic_id=14489

Thanks all.

[PavelGeorgiev]

Fandikurnia,

As you noted yourself, the 'crash' that you observed is really the oom killer - its a mechanism that the linux kernel uses to kill what it thinks is appropriate when you are running out of memory. When that happen (server runs very low on memory) and while the oom killer does its job, the server will be slow and unresponsive and it will be back to normar as soon as some process(es) get killed and some memory is freed.

SO you are dealing with one problem here - you are running out of memory. Either figure out what eats your memory (apache is the prime suspect) and limit its allocation (limit memory per child or number of children), or, if such memory usage is required, then allocate more memory to your appliance.