Welcome, Guest. Please login or register.
Did you miss your activation email?
Tuesday 19 November 2024, 05:28:48 am

Login with username, password and session length

CLICK HERE for the The official Endian Roadmap and Issue tracker
14258 Posts in 4377 Topics by 6515 Members
Latest Member: hulteends
Search:     Advanced search
+  EFW Support
|-+  Support
| |-+  EFW SMTP, HTTP, SIP, FTP Proxy Support
| | |-+  Blacklist Update Script - Download
0 Members and 4 Guests are viewing this topic. « previous next »
Pages: 1 [2] 3 Go Down Print
Author Topic: Blacklist Update Script - Download  (Read 132587 times)
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #15 on: Saturday 28 November 2009, 06:51:57 am »

I get the same error

Code:
root@Firewall:/tmp # ./blacklist_updater.py
: bad interpreter: No such file or directoryon
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #16 on: Saturday 28 November 2009, 06:53:24 am »

Weird...maybe try running it via python directly.

Code:
python blacklist_updater.py
Logged
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #17 on: Saturday 28 November 2009, 06:57:02 am »

excellent, now if it works.  I'll tell you if everything worked. thank you very much for your help

Code:
root@PDH-Firewall:/tmp # python blacklist_updater.py
Blacklist Updater for EFW 2.3
Written by Gavin Vickery <gdvickery@gmail.com>
==============================
Stage 1 of 5 | Downloading blacklist, please wait...
Stage 2 of 5 | Extracting files
Stage 3 of 5 | Updating blacklist
Stage 4 of 5 | Updating categories
Stage 5 of 5 | Cleaning up
==============================
All done! Enjoy ;)
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #18 on: Saturday 28 November 2009, 06:58:24 am »

Glad to be of help  Grin
Logged
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #19 on: Saturday 28 November 2009, 07:03:48 am »

One last question, how often should I perform this update on my server
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #20 on: Saturday 28 November 2009, 07:11:46 am »

Up to you, I usually do it every monday morning at our office. You could safely do it once a month though.
Logged
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #21 on: Saturday 28 November 2009, 07:13:52 am »

Ok, again thanks for your help   Grin Grin Grin
Logged
randle1111
Jr. Member
*
Offline Offline

Posts: 1


« Reply #22 on: Wednesday 06 January 2010, 03:01:30 am »

Thank you very much for this script!  I am using EFW 2.3 and had the same issues as ragr91279.  Running the script as:

python ./blacklist_updater.py

worked!

Some research turned up why a python was needed in front of the script.  I clicked on the original link and then copied the text into notepad (windows).   The dos format puts cr and lf (^M) into the file which causes the program to not run correctly.  I went back and right clicked on the link and selected save as, saved the file (did not open in notepad), transferred the file and then

./blacklist_updater.py

worked!!

I trying to run this script to run automatically.  I tried to put the script in the /var/cron.minutely (will move it to cron.weekly after testing).  It looks like it is working because /var/log/messages is not showing the script exiting error I was getting before (see below)

Job [ -x /bin/run-parts ] && run-parts --report /etc/cron.minutely terminated (exit status: 1)



Oh yea based on the following kb I dropped the .py from the name and put the permissions to 0700:

http://kb.endian.com/entry/43/




If I was dreaming I would ask for help putting the code below into your python script.  The script adds a date updated to your file and then is read by the html to be displayed on the content filter page.

http://efwsupport.com/index.php?topic=4.15

I was able to do the following:

Quote
In /home/httpd/cgi-bin/dansguardian.cgi find:

openbox('100%', 'left', _('URL Blacklist: Filter pages known to have content of the following categories.'), "blacklist");

Replace with this:
$output = `cat  /etc/dansguardian/blacklists/updated`;
$output = &cleanhtml($output,"y");
openbox('100%', 'left', _("URL Blacklist: Filter pages known to have content of the following categories. Update Date: $output"), "blacklist");

Ended up doing the following in dansguardian.cgi, which worked correctly.  I moved the $output lines up a few more lines higher in the file than they recommended, as the output statement did would not work around the get_folding.

Code:

    get_category("phrase_$cat", _(ucfirst($catname)), $allchecked, \%subcategories, \%conf_phraselist);
    $count++;
$output = `cat  /etc/dansguardian/blacklists/updated`;
$output = &cleanhtml($output,"y");
}
printf <<EOF
    </tr></table></td></tr></table>
%s
%s
EOF
,
get_folding(),
get_folding("BLACKLIST", "start", _("Filter pages known to have content of the following categories. (URL Blacklist) Update Date: $output")),
;


I am not good with scripting and the problem I was having was I can not figure out how to add the following to the python script (don't know if you can mix python and sh script).  Perhaps the sh script could be converted to python script:

code that needs to be converted:

Code:
#!/bin/sh
date > /etc/dansguardian/blacklists/updated
exit 0

So to be clear,  I have been able to get the date updated to display correctly by modifying the dansguardian.cgi file.  I can execute the

Code:
date > /etc/dansguardian/blacklists/updated

at the # prompt (using putty). And it displays the date properly in the EFW gui.  But I can not get date >... code to work correctly in the python script so that the date is updated when the script runs successfully.

Please help!  Thanks

p.s.
 I am a newbie in regards to scripting and linux so if I am way off base please forgive me in advance.
Logged
satish
Full Member
***
Offline Offline

Posts: 29


« Reply #23 on: Sunday 10 January 2010, 02:36:17 am »

Alrite ever since i updated, i can no longer white list sites.
As a matter of fact i cant even visit this topic on the site, as the logs say its a porn site.
I tried changing that a/w as mentioned in 1 of the posts. but that hasnt helped.

Anyone care to give a heads up here? (Btw im using 2.3 community)
Logged
jester95
Full Member
***
Offline Offline

Posts: 13


« Reply #24 on: Wednesday 20 January 2010, 06:39:25 am »

Hey EFW Community,

I've written a python script to update the content filter blacklist automatically.

The script downloads the latest copy from shallalist, copies into the dansguardian blacklist directory and updates the categories accordingly.

To use it, scp the "blacklist_update.py" file to the "/tmp" directory on your EFW. Then, go to the "/tmp" directory on your EFW and run the command "./blacklist_updater.py".

I currently use this on Endian Firewall Community release 2.3.0.

Download Here:
http://geekforbrains.com/scripts/blacklist_updater.py

Gavin

The URL for this script no longer exists.  Anybody have a copy or an updated URL?
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #25 on: Friday 22 January 2010, 03:22:39 am »

Hey, I've updated the link. It should work now.

 Wink
Logged
etan15
Jr. Member
*
Offline Offline

Posts: 4


« Reply #26 on: Monday 25 January 2010, 09:51:50 am »

Hi,

I am new to Endian firewalls and was interested in this script but I am getting this error:

root@Endian:/tmp # python ./blacklist_updater.py
Blacklist Updater for EFW 2.3
Written by Gavin Vickery <gdvickery@gmail.com>
==============================
Stage 1 of 5 | Downloading blacklist, please wait...
Traceback (most recent call last):
  File "./blacklist_updater.py", line 18, in ?
    source = urllib2.urlopen(sourceURL)
  File "/usr/lib/python2.4/urllib2.py", line 130, in urlopen
  File "/usr/lib/python2.4/urllib2.py", line 358, in open
  File "/usr/lib/python2.4/urllib2.py", line 376, in _open
  File "/usr/lib/python2.4/urllib2.py", line 337, in _call_chain
  File "/usr/lib/python2.4/urllib2.py", line 1021, in http_open
  File "/usr/lib/python2.4/urllib2.py", line 996, in do_open
urllib2.URLError: <urlopen error (101, 'Network is unreachable')>


Any help is much appreciated!

Thanks in advance...
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #27 on: Tuesday 26 January 2010, 03:20:46 am »

Looks like the firewall cant establish an external connection. Are you sure your network connections are configured correctly? Can you ping external servers?
Logged
etan15
Jr. Member
*
Offline Offline

Posts: 4


« Reply #28 on: Tuesday 26 January 2010, 07:54:40 am »

I am a little confused...  I have no problem getting out to the internet with the box in line so how would I know if  the box was not able to get an external connection?  Here is the the routing table config...

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.16.2.0      0.0.0.0         255.255.255.0   U     0      0        0 br0
172.16.1.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1

Do I have to setup a route manually?

Thanks,
Logged
etan15
Jr. Member
*
Offline Offline

Posts: 4


« Reply #29 on: Tuesday 26 January 2010, 07:58:56 am »

Here are the listed interfaces...

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:10:f3:03:bd:4a brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:10:f3:03:bd:4b brd ff:ff:ff:ff:ff:ff
    inet 172.16.1.2/24 brd 172.16.1.255 scope global eth1
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:10:f3:03:bd:4c brd ff:ff:ff:ff:ff:ff
5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
    link/ether 00:10:f3:03:bd:4a brd ff:ff:ff:ff:ff:ff
    inet 172.16.2.1/24 brd 172.16.2.255 scope global br0

Not very savvy with Linux yet so I am not sure if this is correct...
Logged
Pages: 1 [2] 3 Go Up Print 
« previous next »
Jump to:  

Page created in 0.156 seconds with 19 queries.
Powered by SMF 1.1 RC2 | SMF © 2001-2005, Lewis Media Design by 7dana.com