Print Page - Issue with RRD Graphs and Traffic Monitoring

Title: Issue with RRD Graphs and Traffic Monitoring
Post by: mrkroket on Saturday 21 March 2009, 02:07:13 am

I recently started to test endian firewall.

But a week ago the System & Traffic Graphs started to fail, And I can't see any graph on the web (Proxy graphs keep working).
After searching and googleing I found out that the problem was in /usr/local/bin/rrdfix.sh. With top it appears to take several minutes and more that 10% CPU time to finish.
That script tries to delete all RRD from some dirs, so it shouldn't take too much time:

Code:

#!/bin/sh

DIRS="/var/log/rrd/ /var/ntop/"

rm /home/httpd/html/graphs/*.png &>/dev/null

find $DIRS -name "*.rrd" | \
    while read F; do
    NOW=$(date "+%s")
    LAST=$(rrdtool last $F)
    if [ "$LAST" -gt "$NOW" ]; then
	echo "rrd file $F contains timestamps in future. Remove the file!"
	rm -f $F
    fi
done

The problem is that if you enable some options on Traffic Monitoring, you'll have thousands of rrd files on /var/ntop/rrd/interfaces/br0/hosts/. And that much files takes a lot of CPU time, a lot of HDD usage and a lot of time. That will "time out" the graph refresh. So the main point is to limit the time used with rrdfix.sh.

My dirty solution was to remove the /var/ntop/ from the previous script. System & traffic graphs are in /var/log/rrd/, so I suppose that Traffic Monitoring will handle rrd's in /var/ntop/, and therefore we don't need to delete rrd's from traffic monitoring on that script. Am I wrong with that?

Code:

#!/bin/sh

DIRS="/var/log/rrd/"
# Really needed? /var/ntop/"

rm /home/httpd/html/graphs/*.png &>/dev/null

find $DIRS -name "*.rrd" | \
    while read F; do
    NOW=$(date "+%s")
    LAST=$(rrdtool last $F)
    if [ "$LAST" -gt "$NOW" ]; then
	echo "rrd file $F contains timestamps in future. Remove the file!"
	rm -f $F
    fi
done

Also it's important to limit rrd's on Traffic Monitoring to the hosts you need (https://EFW_IP:3001/editPrefs.html , rrd.hostsFilter option), to reduce the rrd files on /hosts/ folder.

Now the script takes <1 second to finish, CPU usage drop and graph are again displayed.

Check the difference:
(http://img21.imageshack.us/img21/59/cpuweek.png)
From 33.36% CPU time on TUE it drops to 0.39% on FRI. The white bands are when graphs are unavailable due to the rrdfix.sh timing out.

Title: Re: Issue with RRD Graphs and Traffic Monitoring
Post by: jpgillivan on Wednesday 06 May 2009, 12:13:15 am

I haven't experienced any problems like that. But we just implemented EFW a weeks ago. What are your system specs?

My specs:
Processor: P4 2.8GHZ
Memory:
total used free shared buffers cached
Mem: 895592 849716 45876 0 151804 188844
-/+ buffers/cache: 509068 386524
Swap: 1790200 34528 1755672

My memory usage was alot lower but then I changed the cache management to:
Cache management
Harddisk cache size (MB) * 500
Memory cache size (MB) *    200
Max object size (KB) *    1024
Min object size (KB) *    0

Maximum swap space used is 3.6%

Title: Re: Issue with RRD Graphs and Traffic Monitoring
Post by: mrkroket on Thursday 14 May 2009, 11:35:08 pm

I have nothing to do with system specs, it is a bug that time out the rrd graphs.

My specs are a virtualized EFW:
Virtual Guest: EFW
Dual Core
2 GB Ram
40GB HDD

host:
AMD Phenom 8450 (3x Core at 2.10Ghz)
4 GB RAM
600GB HDD

Title: Re: Issue with RRD Graphs and Traffic Monitoring
Post by: inteq on Tuesday 26 May 2009, 11:57:36 am

I post here, so not to start another topic, with related subject.
I do not know what might cause the RRD problem above. Maybe too many clients behind the FW?

My problem is:

I have a constant 4 MB/s upload and most of the time a download of 7-9 MB/s (megabytes) but the graphs show me a max of 550K for one day graph.
Any reason why?
I have to specify that the traffic is made on a forwarded port to a GREEN IP

Title: Re: Issue with RRD Graphs and Traffic Monitoring
Post by: mrkroket on Friday 19 June 2009, 08:18:20 am

Quote from: inteq on Tuesday 26 May 2009, 11:57:36 am

Forwarded inside GREEN? it's an strange setup. The normal Firewall use is traffic going from Green to RED. What you have is a GREEN to GREEN Traffic, that is not graphed.

Title: Re: Issue with RRD Graphs and Traffic Monitoring
Post by: odtabien on Wednesday 02 September 2009, 07:20:54 pm

Quote from: inteq on Tuesday 26 May 2009, 11:57:36 am

I have the same problem with the network traffic graphs. we have a 6Mbps connection but the graph peaks at 600Kbps and i know the usage actually goes at 5.98Mbps during peak hours.

i tried to look for the MRTG configuration but didnt find any, maybe they have used another tool for the graph?

EFW Support

Support => General Support => Topic started by: mrkroket on Saturday 21 March 2009, 02:07:13 am