Greater Blacklist

[Guest]

[hinge]

hi! baggers can you give me a screenshoot what happen to your endian when you try this.. because i did not sacrifice may Endian. actually im beginner this kind of setting and i have scare to try and my endian is only one and i dont have any machine to try this... thank you for your understand.. and thank you for giving an idea.. thank you..

[Baggers]

Hi Hinge,

Which bit are you stuck with and needing clarification on? As long as you make a copy of any files before they are changed you should be fine.... but if you really don't have a 'plan B' if your firewall goes 'up the spout' and would have no immediate access to the resources to fix it then you probably shouldn't try it! Do you know how to restore everything if it did go wrong?

[vondie]

just one question... if you enable the whole "A/D/U/L/T" filter on the "URL Blacklist" part of Content Filter... is it working? I've try it so many time to make it work but no luck.... i've even evaluate the size of the file of a/d/u/l/t file... it is super huge.... comparing to other url blacklist category or shalla's category....

[jaebird]

Here is a little contribution to this thread.

curl is installed by default with endian...so a quick mod of the script means you don't need wget:

#!/bin/sh
cd /tmp
curl -C - -O http://www.shallalist.de/Downloads/shallalist.tar.gz
tar -xvzf shallalist.tar.gz
cp -r BL/* /etc/dansguardian/blacklists/
rm -r /etc/dansguardian/blacklists/COPYRIGHT
rm -r /etc/dansguardian/blacklists/global_usage
date > /etc/dansguardian/blacklists/updated
/etc/init.d/dansguardian restart
exit 0

My change to the cgi file is below, it puts the update date in the box title.

In /home/httpd/cgi-bin/dansguardian.cgi find:

openbox('100%', 'left', _('URL Blacklist: Filter pages known to have content of the following categories.'), "blacklist");

Replace with this:
$output = `cat  /etc/dansguardian/blacklists/updated`;
$output = &cleanhtml($output,"y");
openbox('100%', 'left', _("URL Blacklist: Filter pages known to have content of the following categories. Update Date: $output"), "blacklist");

Notice the change from single to double quotes around the string.

[entourage]

I just got around to adding this to my Endian 2.2 install and I can confirm it works GREAT!!! 

Thanks for the effort!

BTW, if you're a business, don't forget to sign the Shallalist agreement.  It's free to use and they were very good about getting the info to me.

[Dougie]

Thanks worked for me. Community 2.2.

[whoiam55]

Today while reading up on urlblacklist.com I stuble upon this

Q 10. What do you mean about needing unsorting with DansGuardian?
The blacklist is provided as a gzipped tar file with directories containing text file lists. These lists are now provided in a sorted ordered list. DansGuardian, when reading in the files, will in memory sort the file. It needs to sort them to be able to quickly search them when doing the filtering. However DansGuardian uses the Quick Sort algorithm which is very quick to sort random lists but it will take a very LONG time to sort a list that is already sorted. DansGuardian will eventually start. If it does not then you might have a very old version with a bug. The input URL list being sorted should not affect any script or program and stop it actually working.

Q 11. How do I unsort the blacklist?
Unsorting a single file is very easy. Simply use the 'rl' or 'unsort' unix command on each of the text files. You can find information on these here:
http://ch.tudelft.nl/~arthur/rl/
http://www.vanheusden.com/unsort/

To unsort all the files in the blacklist in one go you could:
cd blacklistdir; find . -type f -exec rl '{}' -o '{}'.tmp \; -exec mv -f '{}'.tmp '{}' \;
This will find all the files in blacklistdir and subdirectories and randomize the lines. It should only take a few seconds if that. If you are using unsort instead of rl you will have to edit the above.

Pretty straight forward I guess.

[g.vecchi]

Hi all!
I'm new of EndianFW and I'm trying to create a better blacklist for Content Filtering.

I'm running EFW Community 2.4.1 and I've modified jaebird script in this way:

Code:

#!/bin/sh
cd /tmp

if [ -a /tmp/blacklists ]
  then
    rm -r /tmp/blacklists
fi

curl "url_to_blacklist" -o /tmp/blacklists.tar.gz
tar -xvzf /tmp/blacklists.tar.gz
mv /etc/dansguardian/blacklists /etc/dansguardian/blacklists.bak
rm /tmp/blacklists/CATEGORIES
mkdir /etc/dansguardian/blacklists
cp -r /tmp/blacklists/* /etc/dansguardian/blacklists/


# Check categories file exists
if [ -f /tmp/blacklists/categories ]; then
  rm /tmp/blacklists/categories
else
  touch /tmp/blacklists/categories
fi

# Create categories file
for i in `ls /etc/dansguardian/blacklists/`; do
  if [ "$i" != "categories" ]; then 
    echo "$i"="$i" >> /tmp/blacklists/categories;
  fi
done

cp /tmp/blacklists/categories /etc/dansguardian/blacklists/
rm -r /tmp/blacklists*
chmod 755 -R /etc/dansguardian/blacklists
chmod 644 /etc/dansguardian/blacklists/categories
/etc/init.d/dansguardian restart
exit 0

url_to_blacklist = bigblacklist url  <--- sorry for that but forum doen't allow me to post clickable link



but dansguardian failed to start because it says that

/etc/dansguardian/blacklists/violence/expressions

doesn't exist

So, I've modified /etc/dansguardian/profiles/1/bannedregexpurllist removing line about /etc/dansguardian/blacklists/violence/expressions and dansguardian restart to work: is that a good idea???

Thanks to all!

[portucale]

thanks to g.vecchi and addicional info i've read in other foruns, my script looks like this:

(create the file update_blacklist.sh and save it in /etc/cron.weekly/)

Code:

#!/bin/sh

cd /tmp

if [ -a /tmp/shallalist.tar.gz ]
  then
    rm -r /tmp/shallalist.tar.gz
fi

curl -C - -O SITE

if [ -a /tmp/BL ]
  then
    rm -r /tmp/BL
fi

tar -xvzf shallalist.tar.gz

if [ -a /etc/dansguardian/blacklists.bak ]
  then
    rm -r /etc/dansguardian/blacklists.bak
fi

mv /etc/dansguardian/blacklists /etc/dansguardian/blacklists.bak

mkdir /etc/dansguardian/blacklists/

cp -r /tmp/BL/* /etc/dansguardian/blacklists/

rm -r /etc/dansguardian/blacklists/COPYRIGHT

rm -r /etc/dansguardian/blacklists/global_usage

date > /etc/dansguardian/blacklists/updated

# Create categories file
for i in `ls /etc/dansguardian/blacklists/`; do
  if [ "$i" != "categories" ]; then 
    echo "$i"="$i" >> /etc/dansguardian/blacklists/categories;
  fi
done

/etc/init.d/dansguardian restart

if [ -a /tmp/shallalist.tar.gz ]
  then
    rm -r /tmp/shallalist.tar.gz
fi

if [ -a /tmp/BL ]
  then
    rm -r /tmp/BL
fi

exit 0

IMPORTANT

replace SITE in the line
curl -C - -O SITE
with a blacklist file like shallalist.tar.gz

then in the file /etc/dansguardian/profiles/1/bannedregexpurllist remove the line with /etc/dansguardian/blacklists/violence/expressions
save the file and in shell do
(assuming the file you've create the file update_blacklist.sh in /etc/cron.weekly/

Code:

cd /etc/cron.weekly/
chmod 755 update_blacklist.sh
./update_blacklist.sh

this will execute the script
go to endian->proxy->contentfilter->edit
and in the tab Filter pages known to have content of the following categories. (URL Blacklist)
select the categories you want to block

after that you can restart dansguardian doing in shell

Code:

/etc/init.d/dansguardian restart

all should all get's ok!