EFW Support
Welcome,
Guest
. Please
login
or
register
.
Did you miss your
activation email?
Thursday 28 March 2024, 08:03:41 pm
1 Hour
1 Day
1 Week
1 Month
Forever
Login with username, password and session length
Visit the official Endian Community Mailinglist
HERE
14247
Posts in
4376
Topics by
6489
Members
Latest Member:
GB-gattoboy
Search:
Advanced search
EFW Support
Support
EFW SMTP, HTTP, SIP, FTP Proxy Support
Squint Reports
0 Members and 1 Guest are viewing this topic.
« previous
next »
Pages:
1
2
[
All
]
Author
Topic: Squint Reports (Read 35437 times)
nmatese
Full Member
Offline
Posts: 27
Squint Reports
«
on:
Tuesday 23 February 2010, 02:16:46 am »
Has anyone been able to integrate Squint with Endian? I am having trouble with it, I have it installed and running every night but for some reason it only keeps 1 day of history reports. Is there anyone who knows how to keep it accumulating data more than 1 day period.
Thanks,
Nick
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #1 on:
Saturday 27 March 2010, 02:32:24 am »
I just installed it also because I liked the depth of info much better than other log parsers, but I noticed that it was only getting 1 day.
Basically what I found was the the access.log is being reset/archived each night (as it's supposed to) That's why you have so many .gz files in your /var/log/squid folder.
So unless there's a way to turn off archiving (which would probably break, or render useless, the native log viewer) or have it create a separate log file that isn't reset each evening.
Any ideas anyone?
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #2 on:
Tuesday 06 April 2010, 01:40:20 am »
Ok, so I'm getting close. The command necessary to complete this is:
Code:
cat /var/log/squid/access.log >> /var/log/squid/squint/access.log
Then in your squint.cron.sh you can change the LOGDIR="/var/log/squid/squint"
That way it's looking in the ../squint directory for the new access.log file which is compiling each night.
What I haven't figured out is how to get this to run as a cron job. I'd like to have it run right before the logrotate which happens at 11:57pm each evening. The code appends the current access.log to the end of the new access.log and then squint reads that file.
Hope that helps someone. Feel free to correct anything I've gotten wrong to this point.
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #3 on:
Wednesday 07 April 2010, 05:22:13 am »
Ok, bit of an update...
After much testing I figured out that squint is looking at the 'access.log_short' file instead of 'access.log'
First I created a new folder called 'squint' in the /var/log/squid directory
Next I modified my /usr/local/bin/squint.cron.sh file to reflect the new LOGDIR value:
Code:
LOGDIR="/var/log/squid/squint"
Then I created a file in /etc/cron.d called 'logsquint' (Doesn't matter the name) that contains:
Code:
30 23 * * * /bin/cat /var/log/squid/access.log_short >> /var/log/squid/squint/access.log_short
What this will do is at 11:30pm it will append the current access.log_short to the end of the new access.log_short file in the new squint folder.
Hopefully, and this has yet to be tested, the daily, weekly and monthly cron jobs will run approprately since they were generated by the squint install. Here's the output of my fcrontab -l:
Code:
15:19:45 listing root's fcrontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=
HOME=/
#
&nolog */1 * * * * [ -x /bin/run-parts ] && run-parts --report /etc/cron.minutely
&nolog */5 * * * * [ -x /bin/run-parts ] && run-parts --report /etc/cron.cyclic
01 * * * * [ -x /bin/run-parts ] && run-parts --report /etc/cron.hourly
25 1 * * * [ -x /bin/run-parts ] && run-parts --report /etc/cron.daily
47 2 * * 0 [ -x /bin/run-parts ] && run-parts --report /etc/cron.weekly
52 3 1 * * [ -x /bin/run-parts ] && run-parts --report /etc/cron.monthly
#
@nolog 5 [ -x /bin/run-parts ] && run-parts --report /etc/anacron.cyclic
@ 1h [ -x /bin/run-parts ] && run-parts --report /etc/anacron.hourly
@ 1d [ -x /bin/run-parts ] && run-parts --report /etc/anacron.daily
@ 1w [ -x /bin/run-parts ] && run-parts --report /etc/anacron.weekly
@ 1m [ -x /bin/run-parts ] && run-parts --report /etc/anacron.monthly
# Summarize ip accounting info:
2 1 * * * /usr/sbin/ipacsum -r -H `/bin/hostname` -t "the day 2 days ago" >/dev/null
3 1 * * 0 /usr/sbin/ipacsum -r -H `/bin/hostname` -t "the week 4 weeks ago" >/dev/null
4 1 1 2 * /usr/sbin/ipacsum -r -H `/bin/hostname` -t "the year 2 years ago" >/dev/null
# squint squid reports
# Weekly, on Mondays
00 01 * * Monday root /usr/local/bin/squint.cron.sh weekly
# Monthly, on the first day of the month
00 02 1 * * root /usr/local/bin/squint.cron.sh monthly
# Daily at 3am
00 03 * * * root /usr/local/bin/squint.cron.sh daily
57 23 * * * [ -x /usr/lib/cron/logrotate.cron ] && /usr/lib/cron/logrotate.cron
00 15 * * * /bin/cat /var/log/squid/access.log_short >> /var/log/squid/squint/access.log_short
00 23 * * * [ -x /usr/lib/cron/sargdaily.cron ] && /usr/lib/cron/sargdaily.cron
I'll post back tomorrow with hopefully positive results!
Logged
gyp_the_cat
Full Member
Offline
Posts: 81
Re: Squint Reports
«
Reply #4 on:
Wednesday 07 April 2010, 05:29:09 am »
Hi Entourage, wow
Looking forward to what you manage to get working
Logged
nmatese
Full Member
Offline
Posts: 27
Re: Squint Reports
«
Reply #5 on:
Wednesday 07 April 2010, 05:30:27 am »
This does appear to be working, I am also testing it this evening.
Logged
gyp_the_cat
Full Member
Offline
Posts: 81
Re: Squint Reports
«
Reply #6 on:
Wednesday 07 April 2010, 05:32:40 am »
In that case nmatese I'm looking forward to both your findings
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #7 on:
Wednesday 07 April 2010, 06:58:32 am »
The only downside to this (so far) is that there isn't a live report.
You can create a once-off report to view current stats:
Code:
cat /var/log/squid/access.log_short | /usr/local/bin/squint.pl /home/httpd/html/instant 0 1270590423
Then browse to http://<ip>/instant
*Note: The 1270590423 is unix time. I haven't experimented with other time formats, but you have to have some sort of date range in there (in my case 0 which is Jan 1, 1970 to 1270590423 which is April 6, 2010 @ 4:47pm.
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #8 on:
Wednesday 07 April 2010, 11:01:03 pm »
Well...it appears that everything worked last night, except I forgot the change my cat cron job to 11:30pm. So it didn't copy over my access.log_short. I just needed one more reboot. DANG
Oh well...try it again tonight!
How did things fair for you nmatese?
Logged
nmatese
Full Member
Offline
Posts: 27
Re: Squint Reports
«
Reply #9 on:
Thursday 08 April 2010, 04:27:43 am »
Everything seems to have worked on my end, the best test will be this evening when a second days worth of data is added. That is where we ran into trouble before, so I will report back again tomorrow AM.
Thanks,
Nick
Logged
quickben
Jr. Member
Offline
Posts: 8
Re: Squint Reports
«
Reply #10 on:
Thursday 08 April 2010, 05:21:15 am »
This sounds exactly like what I need! Any chance of a step by step walk through on installing and configuring?
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #11 on:
Thursday 08 April 2010, 06:20:30 am »
The basic installation of squint is relatively easy and included in the INSTALL file included with the gz file.
First, grab the squint.tar.gz file from the site:
http://www.ledge.co.za/software/squint/squint.tar.gz
If you don't already have it, I recommend getting WinSCP to transfer the squint.tar.gz file to a location on your Endian. For me, I just copied it to the /root folder.
Here's a copy of the Installation text:
Quote
INSTALLATION
# tar -zxvf squint.tar.gz
I hope this part is easy enough
# cp squint.pl squint.cron.sh /usr/local/bin
Install the binaries in a nice place
# squint.cron.sh init
This creates a place for the reports in
/usr/local/httpd/htdocs (or whereever the htdocs
directory is found). It installs a crontab entry to run
the report daily, weekly and monthly. You may wish to
edit the crontab entry first -- the system has not been
optimised for security, and runs (unnecessarily) as the
user root, rather than the user squid.
Note that if you run squint.cron.sh at all, it is likely to make
crontab entries for you (!)
# squint.cron.sh all
Create current daily, weekly and monthly reports -- don't
do this twice! -- the reports get rotated every time you
run the task, so things can get a little messy.
# squint.cron.sh regen
Recreate all the reports for your initial system, and
also for when the program manges to mess things up. You
need to run this to maintain weekly and monthly reports
if your server happens to miss a cron job.
Once you've followed those initial instructions, find my post above about creating the squint folder and then what to modify in the /usr/local/bin/squint.cron.sh file
All in all, it's not bad. Let me know if I missed something.
Logged
quickben
Jr. Member
Offline
Posts: 8
Re: Squint Reports
«
Reply #12 on:
Thursday 08 April 2010, 08:55:00 am »
Brilliant! Thanks for that and will post back if it goes pear shaped. I'm also looking at bandwidthd as a solution and should get to try it this weekend on a VM's. Will start a new thread on that topic.
Logged
nmatese
Full Member
Offline
Posts: 27
Re: Squint Reports
«
Reply #13 on:
Thursday 08 April 2010, 11:42:30 pm »
Im happy to report that this is working as intended for me. It is keeping track of multiple days of logs. Thanks so much for your investigation entourage!
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #14 on:
Thursday 08 April 2010, 11:46:23 pm »
Excellent! Glad to have helped! Mine seems to be working well also!
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #15 on:
Tuesday 13 April 2010, 01:40:32 am »
Ok, might have found a glitch...
Was hoping to see the new 'weekly' report show up which was scheduled to run Monday morning at 1:00am, however I think there's an error in fcrontab:
Code:
00 01 * * Monday root /usr/local/bin/squint.cron.sh weekly
I don't know if it matters or not, but I think that instead of 'Monday' it was supposed to be '1' so the whole line should have read:
Code:
00 01 * * 1 root /usr/local/bin/squint.cron.sh weekly
If anyone knows how to change it, let me know because my attempts to edit fcrontab have yet-so-far failed.
Logged
entourage
Full Member
Offline
Posts: 48
Re: Squint Reports
«
Reply #16 on:
Thursday 28 October 2010, 02:52:57 am »
So everything was going along nicely, until my root partition was filled up with a 10GB log file.
Yes, I knew that could happen eventually, but didn't know how fast...
Anyway, what I would like to do is purge anything from the log that has a timestamp older than Aug 1st 2010 - Unix timestamp: 1280638801
Anyone have ANY ideas how to do that??
Logged
Pages:
1
2
[
All
]
« previous
next »
Jump to:
Please select a destination:
-----------------------------
Announcements
-----------------------------
=> Project News
=> Latest News and Updates
-----------------------------
Support
-----------------------------
=> General Support
=> Installation Support
=> EFW SMTP, HTTP, SIP, FTP Proxy Support
=> VPN Support
=> Hardware Support
-----------------------------
Development
-----------------------------
=> EFW Wishlist
=> Contribute Your Customisations & Modifications
Page created in 0.094 seconds with 18 queries.
Powered by SMF 1.1 RC2
|
SMF © 2001-2005, Lewis Media
Design by
7dana.com