Blacklist Update Script - Download

[Guest]

[Gavin]

Hey EFW Community,

I've written a python script to update the content filter blacklist automatically.

The script downloads the latest copy from shallalist, copies into the dansguardian blacklist directory and updates the categories accordingly.

To use it, scp the "blacklist_update.py" file to the "/tmp" directory on your EFW. Then, go to the "/tmp" directory on your EFW and run the command "./blacklist_updater.py".

I currently use this on Endian Firewall Community release 2.3.0.

Download Here:
http://geekforbrains.com/scripts/blacklist_updater.py

Gavin

[NinNin]

Thank you very much.   

[Gavin]

My pleasure! 

One thing I forgot to mention is that this script will overwrite the current blacklist. If you want to keep the original, you may want to back it up first. You can do so by running the following command:

Code:

cp -ruv /etc/dansguardian/blacklists /etc/dansguardian/blacklists_orig

[sepp]

hi,
when I use your script to update the Endian, the pages wouldn't be filtered exactly. For example in the Category "s" there is a entry for "youtube.com" and "youtube.com" is filtered but "youtube.com/de" would't be filtered. Is there something wrong in the script ?

Is it possible that Endian only works with the "urls-file" and not with the "domains" file ?

[Gavin]

Hi sepp,

I had the exact same problem you are. Its a bug in the restart file for dansguardian. The file is located at:

Code:

/usr/local/bin/restartdansguardian.py

Here is a cut-out of that file:

Code:

#exceptionsitefile = open("/etc/dansguardian/profiles/%s/exceptionsitelist" % number, "w")
exceptionsitefile = open("/etc/dansguardian/profiles/%s/exceptionsitelist" % number, "a")
exceptionsitefile.write(exceptionsitelist)
exceptionsitefile.close()

bannedsitelist = ""
if file_exists("%s/bannedsitelist" % path):
    bannedsitefile = open("%s/bannedsitelist" % path, "r")
    bannedsitelist = bannedsitefile.read()
    bannedsitefile.close()

#bannedsitefile = open("/etc/dansguardian/profiles/%s/bannedsitelist" % number, "w")
bannedsitefile = open("/etc/dansguardian/profiles/%s/bannedsitelist" % number, "a")
bannedsitefile.write(bannedsitelist)
bannedsitefile.close()

The lines I've commented out are the originals. Basically you're replacing the "w" with "a", which means append instead of write. The lines start around line 308.

[mrkroket]

Two suggestions:

1- Put a link of that post on Customization forum, to group all custom mods/scripts together
2- A button on the HTTP Proxy page that exec this script. This way one can run the updater from the GUI, without much hassle

[nmatese]

I really appreciate the update, this is great add-on.

[ragr91279]

Hi, I have tried using your script, but I receive the following error   

: bad interpreter: No such file or directory

in the / tmp folder I copied the script

What am I doing wrong?

[Gavin]

Sounds like its having an issue finding Python. What version of EFW are you running?

Try running the command:

Code:

whereis python

My output is:

Code:

python: /usr/bin/python2.4 /usr/bin/python /usr/lib/python2.4

[ragr91279]

I have version 2.3, I will try what you say

[ragr91279]

Sounds like its having an issue finding Python. What version of EFW are you running?

Try running the command:

Code:

whereis python

My output is:

Code:

python: /usr/bin/python2.4 /usr/bin/python /usr/lib/python2.4

I ran the command you say and the output is equal to yours

python: /usr/bin/python2.4 /usr/bin/python /usr/lib/python2.4

[Gavin]

Hmm, how are you running the script? Is it executable?

Try running:

Code:

chmod u+x blacklist_updater.py

Then re-run the script:

Code:

./blacklist_updater.py

Does it work now?

[ragr91279]

Hmm, how are you running the script? Is it executable?

Try running:

Code:

chmod u+x blacklist_updater.py

Then re-run the script:

Code:

./blacklist_updater.py

Does it work now?

I'm afraid not, still shows the same output as before

root@Firewall:/tmp # ./blacklist_updater.py

: bad interpreter: No such file or directoryon

[ragr91279]

if I run the command like this:

Code:

bash ./blacklist_updater.py

  this is the output:

Code:

root@Firewall:/tmp # bash ./blacklist_updater.py
./blacklist_updater.py: line 2: import: command not found
./blacklist_updater.py: line 3: import: command not found
: command not found.py: line 4:
./blacklist_updater.py: line 5: print: command not found
./blacklist_updater.py: line 6: print: command not found
./blacklist_updater.py: line 7: print: command not found
: command not found.py: line 8:
./blacklist_updater.py: line 9: sourceURL: command not found
./blacklist_updater.py: line 10: sourceFile: command not found
./blacklist_updater.py: line 11: blacklistDir: command not found
./blacklist_updater.py: line 12: categoryFile: command not found
: command not found.py: line 13:
./blacklist_updater.py: line 17: print: command not found
./blacklist_updater.py: line 18: syntax error near unexpected token `('
'/blacklist_updater.py: line 18: `source = urllib2.urlopen(sourceURL)

[Gavin]

Remove "bash" from the command. Your executing a python script, not a bash script.

Code:

root@Firewall:/tmp ./blacklist_updater.py