Greater Blacklist

[Guest]

[finchwizard]

I'm still working on it, but here's a screenshot of what I have so far.

Hopefully I'll release a script, or at least manual instructions for it.

But the current Blacklist is very minimal and I personally need far more blacklists. So far mine is looking like this and seems to be working.



Will post back when I get it all working.

[euser4life]

This is probably widely known but for some like myself it is good information.  You can easily update dansguardian blacklist within endian firewall with the newest list of sites / categories by downloading the list from:

<http://urlblacklist.com/?sec=download>

Then (for windows user's use 7-zip to decompress into a folder called blacklists) and use a SFTP client (such as WinSCP) to upload the list to the /etc/dansguardian/blacklists directory.  Once you refresh the page new categories will be there for you to choose from. 

I'm sure there are ways to automate (script) updates.  The first download is free, they don't block additional downloads but go off of the honor system since they are providing a service.  It works very well.

[Hinze57]

Can you post at least some raw instructions?

I can get the blacklists but am trying to figure out if I need to update the categories file. 

Thanks,
K

[wharfratjoe]

Does this work on 2.2 RC3?

[zman818]

I second that question.  It seems that the categories file format is different between the current EFW and current blacklist files...    ?

[khhkom]

any update on this? even after just replacing the domains/urls of some categories dansguardian is stuck with 100% cpu

[hinge]

Itry this configuration to my content filter but my network cannot connect any website? they have any configuration i can do? i using 2.2 rc3.. thank you

[davvidde]

I confirm: the big blacklist from urlblacklist.com do not works because the CPU got 100% for over two hours on a VM (1024MB) on HP proliant QuadCore 2.50GHz  with VMWare ESXi 3.5 and at the end the dansguardian process do not start (terminated with segmentation fault). I try this on EFW 2.2rc3 and EFW2.2 final. I even substitute the /etc/dansguardian/blacklists/CATEGORIES provided by urlblacklist.com with the original provided by endian (the two formats are a bit different) but the problems seems to be the compilation of lists. I even notice the strange behaviour on my virtualbox VM installation: the compilation of the lists do not get over the eleventh. The lists processed are always 11 in total when the categories are 86. This appears on all my three VM on test.
May anyone post dansguardian detailed logs to compare?

[maurelio]

Hi,

I had the same problem in version 2.1.2, but found that the way that files are being provided by the URLBlacklist is incompatible with the Dansguardian.

These files are indexed alphabetically and Dansguardian to work with does not know this.

Just to give you rearrange the files to a random order.

I would like to know how to do this update in version 2.2 final, because the formats are different and the obstacles that exist by default are very old.

[]´s

[davvidde]

I know of this "bad" implementation of Dansguardian which use a quicksort algorithm to sort an already sorted list but I don't understand how Dansguardian in Ubuntu 8.04 processes without problem, in less than a minute, the list provided by URLBLACKLIST.COM; so i decided to copy the already  sorted from Ubuntu to Endian and Dansguardian restart without problem in few seconds. By the way I tested this in EFW2.2 final but this release suffers, in my testing, of authentication problem with LDAP that EFW2.2rc3 does not have. See post http://efwsupport.com/index.php?topic=673.0
Windows authentication in EFW 2.2 also do not work with group policies but I don'k know if this is by design.

[entourage]

I ran into the same problem that you guys had with the URLBLACKLIST.COM killing my install.  I was wondering if Shalla's list http://www.shallalist.de/ would work any better.  It would be nice if wget were installed, so we could run the scripts.
I'm going to manually download the list and move the folders over via WinSCP.  Wish me luck!

[entourage]

Ok, I've downloaded and unzipped the Shalla list.
I removed all of the current folders in /etc/dansguardian/blacklists and copied over the new unzipped folders. 
I then modified the categories file to reflect the new folders.  I did it the quick and dirty way where they each have their own category:

Code:

Adv=adv
Agressive=agressive
Automobile=automobile
Chat=chat
CostTraps=costtraps
Downloads=downloads
etc...

I refreshed the Content Filter page and selected 10 or so categories to block.  Once I hit 'Save' I would say it took approximately 2 minutes to refresh, but it clocked my selected categories.  I noticed I missed a , so I selected about 5 more and hit 'Save' again.  This time it was nearly instant.

So as for manually up, this will work for me, but an automated script would be MUCH nicer!

Hope that helps someone.

[Baggers]

Thanks, I've just automated this by firstly getting wget onto my box based on clues from here...

http://www.howtoforge.com/install-vmware-tools-on-endian-firewall-community-2.2-rc3-on-esxi-3.5

....and then writing the following script and sticking it in /etc/cron.weekly....

#!/bin/sh
wget -Y no http://www.shallalist.de/Downloads/shallalist.tar.gz -O /tmp/shallalist.tar.gz
cd /tmp
tar -xvzf shallalist.tar.gz
cp -r BL/* /etc/dansguardian/blacklists/
rm -r /etc/dansguardian/blacklists/COPYRIGHT
rm -r /etc/dansguardian/blacklists/global_usage
date > /etc/dansguardian/blacklists/updated
/etc/init.d/dansguardian restart
exit 0


... then I modified  /home/httpd/cgi-bin/dansguardian.cgi  so its displays when the blacklist has been updated in the proxy / content filter section by adding....

$output = `cat  /etc/dansguardian/blacklists/updated`;
$output = &cleanhtml($output,"y");
print "URL Blacklist Date : $output\n";



Hope someone finds this useful!

Baggers

[hinge]

hi! baggers.. this is work to EFW community 2.2? thank you..

[Baggers]

Yes. Thats the version i'm using.