EFW Support

I'm running Endian 2.3 and it constantly has the CPU running at close to 100%.  I've checked top and there is no process using that amount but the system usage is high.

Tasks:  59 total,   4 running,  55 sleeping,   0 stopped,   0 zombie
Cpu(s): 12.0%us, 76.4%sy,  0.0%ni,  0.0%id, 10.3%wa,  0.4%hi,  0.9%si,  0.0%st

Anyone else having the same problem?
Any idea what would be causing the system usage to be so high?

Check with "top" what process got the near 100% CPU, it is impossible that is no processes gain cpu time.

None :S  I don't get it...Again, if you look the CPU usage is sitting at over 90% but the processes only add up to 12% CPU utilization...

top - 18:47:14 up 8 days,  3:27,  1 user,  load average: 2.11, 1.85, 1.60
Tasks:  59 total,   3 running,  56 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.7%us, 86.3%sy,  0.0%ni,  0.0%id, 10.7%wa,  0.3%hi,  0.0%si,  0.0%st
Mem:    254908k total,   250876k used,     4032k free,    31364k buffers
Swap:   514072k total,       72k used,   514000k free,   152668k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+    TIME COMMAND
26168 root      25   0  2096  560  432 R 10.3  0.2   1:13.49   1:13 rrdfix.sh
26167 root      18   0  1532  512  412 S  0.7  0.2   0:05.95   0:05 find
    1 root      16   0  1512  568  480 S  0.0  0.2   0:18.65   0:18 init
    2 root      11  -5     0    0    0 S  0.0  0.0   0:00.01   0:00 kthreadd
    3 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00   0:00 migration/0
    4 root      34  19     0    0    0 S  0.0  0.0   0:09.47   0:09 ksoftirqd/0
    5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00   0:00 watchdog/0
    6 root      10  -5     0    0    0 S  0.0  0.0   0:00.97   0:00 events/0
    7 root      11  -5     0    0    0 S  0.0  0.0   0:00.10   0:00 khelper
   42 root      10  -5     0    0    0 S  0.0  0.0   0:16.74   0:16 kblockd/0
   43 root      20  -5     0    0    0 S  0.0  0.0   0:00.00   0:00 kacpid
   44 root      20  -5     0    0    0 S  0.0  0.0   0:00.00   0:00 kacpi_notify
  210 root      16  -5     0    0    0 S  0.0  0.0   0:00.00   0:00 cqueue/0
  211 root      16  -5     0    0    0 S  0.0  0.0   0:00.00   0:00 ksuspend_usb
  214 root      16  -5     0    0    0 S  0.0  0.0   0:00.00   0:00 khubd
  216 root      11  -5     0    0    0 S  0.0  0.0   0:00.00   0:00 kseriod
  237 root      20   0     0    0    0 S  0.0  0.0   0:00.00   0:00 pdflush

Anyone have any ideas what could be doing this???

*Bump
Surely someone out there knows??
It's starting to cause problems for me with the CPU usage running high constantly.  I also want to enable proxy and cache but with the CPU already maxed out I fear it will reduce my performance rather than increase it :(

It could be something to do with the power management of your CPU

Try this in putty:
echo 1 > /sys/module/processor/parameters/max_cstate


( The default max_cstate is 8 )

Actually, it looks like you don't have enough RAM

256MB is hardly enough to run everything Endian has.
Watch your swap usage, you should never be using swap space.
If you start getting into swap, your system performance is going to go down the drain even if you have super-fast hard disks.

:)

It seems to me the old classic rrdfix.sh error. It was on 2.2, and probably will be in 2.3.

of rrdfix.sh error:
-High CPU usage, dure to rrdfix.sh
-rrdfix.sh execution takes more than 10 seconds.
-Broken graphs on Status tab

Causes:
-rrdfix.sh takes too long to execute. It should be only a few millisecs. But if you have Traffic Monitoring (ntop) enabled, some configurations can hang up the server.

Is your case?
On console, run rrdfix.sh. If it takes too long to finish, you have the problem.


Solutions?:
1- Disable Traffic monitoring (ugh!)
2- Configure Traffic Monitoring to only catch data of your local hosts. eg 192.168.0.0/16
3- Recheck the rrdfix script to find the problem:

This is the rrdfix.sh script:
#!/bin/sh

DIRS="/var/log/rrd/ /var/ntop/ /var/lib/collectd/rrd/"

rm /home/httpd/html/graphs/*.png &>/dev/null

find $DIRS -name "*.rrd" | \
    while read F; do
    NOW=$(date "+%s")
    LAST=$(rrdtool last $F)
    if [ "$LAST" -gt "$NOW" ]; then
        echo "rrd file $F contains timestamps in future. Remove the file!"
        rm -f $F
    fi
done


In my case I have the problem on one of the dirs (maybe /var/ntop?). It has way a lot of dirs and files, takes even minutes to delete all files.

I tried "echo 1 > /sys/module/processor/parameters/max_cstate" was I meant to see something happen?
I thought maybe it changed a system parameter and wouldn't show anything so I rebooted the console and still have high system usage.

I know 256mb isn't a huge amount of RAM, but the swap usage is showing 72k only.  That's not really using it ;)  If I did add extra memory however how would that reduce CPU usage which is my main concern? 

I already had traffic monitoring disabled as I read that it can cause issues with high CPU, but I ran rrdfix anyhow and it hangs after showing:
# rrdfix.sh
ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory
ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory

And after all these Top is still showing 60-80% CPU usage and I'm still pulling at my hair   :'(

How much time rrdfix takes to execute?

Mine takes about 2-3 seconds.

I ran it and it comes up with:


At this point it doesn't return to the command line and just hangs for at least 15mins.  I had to CTRL+C in Putty to return back to command line.

You have it. This is what is bloating your system. Edit the /usr/local/bin/rrdfix.sh script and remove the /var/ntop from the DIRS variable.
Run again the script and see if that reduces your execution time.

Genius!!!  Now execution time is down to 2-3 secs.  CPU usage appears ok at present, I'll monitor it and hopefully it'll work fine :)

24 hours and server is still happy :)  IOU 1 Beer :P

Just wanted to update you kind sir.  It has now been 5 days and CPU usage has been normal.  Thanks for your great help, I can continue to use Endian and be happy :)

Hi,

I have just installed v 2.3 and on the system dash board it constantly show one of my cpu over 95% and other over 65%. When i check ntop it shows following..

Tasks: 134 total,   1 running, 133 sleeping,   0 stopped,   0 zombie
Cpu(s):  7.7%us,  1.2%sy,  0.0%ni, 91.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   2064748k total,   255724k used,  1809024k free,    23244k buffers
Swap:   530136k total,        0k used,   530136k free,    94672k cached

I tried rrdfix and it returns the cursor back in 2 seconds. I also tried "echo 1 > /sys/module/processor/parameters/max_cstate" but nothing happened!!
Is there a problem with gui reporting or ntop is showing wrong statics??

SAM,

I have the same problem.
I have 512 MB of RAM, a CPU of 3.2 Ghz and this must to be more than enough!!.
I deactivated all antivirus check, antispam and any content filter.....and off course, no traffic monitoring.

But when i tried to access through VPN (OpenVPN), the CPU usage is around 100% and the are many errors that packet can't be decrypted and so on. All problems derived from the high ratio of CPU usage.

I execute rrdfix.sh but it takes no time to execute (so fast).
Any ideas?

Now i thinking to downgrade to 2.1 version.......
Thanks in advance and sorry for my english.

use top on console to see what process is eating up your CPU

Yes, it was the first thing i did.

It shows a 0.3% User consume and a 8.3% Sys consume, and the toppest program is openvpn with an average of 16%.
Next, but with an average of 1% is collectd (?).

But, when i display  the system grafics with the gui it shows a cpu charge of 70% and high.
What happend?

I have resolve the cuestion but there is something  strange.

I enabled the VPN Firewall and the CPU usage drop down to 2%.

The strange thing is, before i enabled the VPN Firewall, when there was low usage of users, i could connect to the network (using a high ratio of CPU).......and i think that there should not be possible if i didn't configure the VPN firewall rules, like i did now.

I hope this can help anybody.
Regards