Welcome, Guest. Please login or register.
Did you miss your activation email?
Friday 06 December 2024, 06:06:14 am

Login with username, password and session length

CLICK HERE for the The official Endian Roadmap and Issue tracker
14261 Posts in 4377 Topics by 6517 Members
Latest Member: Sandro
Search:     Advanced search
+  EFW Support
|-+  Support
| |-+  EFW SMTP, HTTP, SIP, FTP Proxy Support
| | |-+  Blacklist Update Script - Download
0 Members and 1 Guest are viewing this topic. « previous next »
Pages: [1] 2 3 Go Down Print
Author Topic: Blacklist Update Script - Download  (Read 144191 times)
Gavin
Full Member
***
Offline Offline

Posts: 20


« on: Friday 13 November 2009, 02:43:29 am »

Hey EFW Community,

I've written a python script to update the content filter blacklist automatically.

The script downloads the latest copy from shallalist, copies into the dansguardian blacklist directory and updates the categories accordingly.

To use it, scp the "blacklist_update.py" file to the "/tmp" directory on your EFW. Then, go to the "/tmp" directory on your EFW and run the command "./blacklist_updater.py".

I currently use this on Endian Firewall Community release 2.3.0.

Download Here:
http://geekforbrains.com/scripts/blacklist_updater.py

Gavin
Logged
NinNin
Full Member
***
Offline Offline

Posts: 24


« Reply #1 on: Friday 13 November 2009, 11:39:45 pm »

Thank you very much.   Smiley
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #2 on: Saturday 14 November 2009, 03:00:06 am »

My pleasure!  Grin

One thing I forgot to mention is that this script will overwrite the current blacklist. If you want to keep the original, you may want to back it up first. You can do so by running the following command:

Code:
cp -ruv /etc/dansguardian/blacklists /etc/dansguardian/blacklists_orig
Logged
sepp
Jr. Member
*
Offline Offline

Posts: 1


« Reply #3 on: Tuesday 17 November 2009, 08:32:39 pm »

hi,
when I use your script to update the Endian, the pages wouldn't be filtered exactly. For example in the Category "s" there is a entry for "youtube.com" and "youtube.com" is filtered but "youtube.com/de" would't be filtered. Is there something wrong in the script ?

Is it possible that Endian only works with the "urls-file" and not with the "domains" file ?
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #4 on: Wednesday 18 November 2009, 02:54:50 am »

Hi sepp,

I had the exact same problem you are. Its a bug in the restart file for dansguardian. The file is located at:

Code:
/usr/local/bin/restartdansguardian.py

Here is a cut-out of that file:

Code:
#exceptionsitefile = open("/etc/dansguardian/profiles/%s/exceptionsitelist" % number, "w")
exceptionsitefile = open("/etc/dansguardian/profiles/%s/exceptionsitelist" % number, "a")
exceptionsitefile.write(exceptionsitelist)
exceptionsitefile.close()

bannedsitelist = ""
if file_exists("%s/bannedsitelist" % path):
    bannedsitefile = open("%s/bannedsitelist" % path, "r")
    bannedsitelist = bannedsitefile.read()
    bannedsitefile.close()

#bannedsitefile = open("/etc/dansguardian/profiles/%s/bannedsitelist" % number, "w")
bannedsitefile = open("/etc/dansguardian/profiles/%s/bannedsitelist" % number, "a")
bannedsitefile.write(bannedsitelist)
bannedsitefile.close()

The lines I've commented out are the originals. Basically you're replacing the "w" with "a", which means append instead of write. The lines start around line 308.
Logged
mrkroket
Hero Member
*****
Offline Offline

Posts: 495


« Reply #5 on: Wednesday 18 November 2009, 04:42:21 am »

Two suggestions:

1- Put a link of that post on Customization forum, to group all custom mods/scripts together
2- A button on the HTTP Proxy page that exec this script. This way one can run the updater from the GUI, without much hassle
Logged
nmatese
Full Member
***
Offline Offline

Posts: 27


« Reply #6 on: Friday 20 November 2009, 04:51:19 am »

I really appreciate the update, this is great add-on.
Logged
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #7 on: Saturday 28 November 2009, 05:18:57 am »

Hi, I have tried using your script, but I receive the following error   

: bad interpreter: No such file or directory

in the / tmp folder I copied the script

What am I doing wrong?

Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #8 on: Saturday 28 November 2009, 05:44:53 am »

Sounds like its having an issue finding Python. What version of EFW are you running?

Try running the command:
Code:
whereis python

My output is:
Code:
python: /usr/bin/python2.4 /usr/bin/python /usr/lib/python2.4
Logged
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #9 on: Saturday 28 November 2009, 06:25:02 am »

I have version 2.3, I will try what you say
Logged
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #10 on: Saturday 28 November 2009, 06:28:30 am »

Sounds like its having an issue finding Python. What version of EFW are you running?

Try running the command:
Code:
whereis python

My output is:
Code:
python: /usr/bin/python2.4 /usr/bin/python /usr/lib/python2.4

I ran the command you say and the output is equal to yours

python: /usr/bin/python2.4 /usr/bin/python /usr/lib/python2.4
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #11 on: Saturday 28 November 2009, 06:31:14 am »

Hmm, how are you running the script? Is it executable?

Try running:
Code:
chmod u+x blacklist_updater.py

Then re-run the script:
Code:
./blacklist_updater.py

Does it work now?
Logged
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #12 on: Saturday 28 November 2009, 06:38:25 am »

Hmm, how are you running the script? Is it executable?

Try running:
Code:
chmod u+x blacklist_updater.py

Then re-run the script:
Code:
./blacklist_updater.py

Does it work now?

I'm afraid not, still shows the same output as before

root@Firewall:/tmp # ./blacklist_updater.py

: bad interpreter: No such file or directoryon


Logged
ragr91279
Jr. Member
*
Offline Offline

Posts: 9


« Reply #13 on: Saturday 28 November 2009, 06:44:52 am »

if I run the command like this:
Code:
bash ./blacklist_updater.py
  this is the output:

Code:
root@Firewall:/tmp # bash ./blacklist_updater.py
./blacklist_updater.py: line 2: import: command not found
./blacklist_updater.py: line 3: import: command not found
: command not found.py: line 4:
./blacklist_updater.py: line 5: print: command not found
./blacklist_updater.py: line 6: print: command not found
./blacklist_updater.py: line 7: print: command not found
: command not found.py: line 8:
./blacklist_updater.py: line 9: sourceURL: command not found
./blacklist_updater.py: line 10: sourceFile: command not found
./blacklist_updater.py: line 11: blacklistDir: command not found
./blacklist_updater.py: line 12: categoryFile: command not found
: command not found.py: line 13:
./blacklist_updater.py: line 17: print: command not found
./blacklist_updater.py: line 18: syntax error near unexpected token `('
'/blacklist_updater.py: line 18: `source = urllib2.urlopen(sourceURL)
Logged
Gavin
Full Member
***
Offline Offline

Posts: 20


« Reply #14 on: Saturday 28 November 2009, 06:48:01 am »

Remove "bash" from the command. Your executing a python script, not a bash script.

Code:
root@Firewall:/tmp ./blacklist_updater.py
Logged
Pages: [1] 2 3 Go Up Print 
« previous next »
Jump to:  

Page created in 0.156 seconds with 19 queries.
Powered by SMF 1.1 RC2 | SMF © 2001-2005, Lewis Media Design by 7dana.com