Welcome, Guest. Please login or register.
Did you miss your activation email?
Saturday 14 December 2024, 12:58:28 am

Login with username, password and session length

CLICK HERE for the The official Endian Roadmap and Issue tracker
14261 Posts in 4377 Topics by 6517 Members
Latest Member: Sandro
Search:     Advanced search
+  EFW Support
|-+  Development
| |-+  Contribute Your Customisations & Modifications
| | |-+  Greater Blacklist
0 Members and 3 Guests are viewing this topic. « previous next »
Pages: 1 [2] Go Down Print
Author Topic: Greater Blacklist  (Read 164518 times)
hinge
Full Member
***
Offline Offline

Posts: 93


« Reply #15 on: Tuesday 04 August 2009, 06:15:06 pm »

hi! baggers Smiley can you give me a screenshoot what happen to your endian when you try this.. because i did not sacrifice may Endian. actually im beginner this kind of setting and i have scare to try and my endian is only one and i dont have any machine to try this... thank you for your understand.. and thank you for giving an idea.. thank you..
Logged
Baggers
Jr. Member
*
Offline Offline

Posts: 4


« Reply #16 on: Wednesday 05 August 2009, 04:03:05 am »

Hi Hinge,

Which bit are you stuck with and needing clarification on? As long as you make a copy of any files before they are changed you should be fine.... but if you really don't have a 'plan B' if your firewall goes 'up the spout' and would have no immediate access to the resources to fix it then you probably shouldn't try it! Do you know how to restore everything if it did go wrong?

Logged
vondie
Full Member
***
Offline Offline

Posts: 22


« Reply #17 on: Saturday 15 August 2009, 11:12:13 pm »

just one question... if you enable the whole "A/D/U/L/T" filter on the "URL Blacklist" part of Content Filter... is it working? I've try it so many time to make it work but no luck.... i've even evaluate the size of the file of a/d/u/l/t file... it is super huge.... comparing to other url blacklist category or shalla's category....

Logged
jaebird
Jr. Member
*
Offline Offline

Posts: 2


« Reply #18 on: Sunday 16 August 2009, 03:45:10 am »

Here is a little contribution to this thread.

curl is installed by default with endian...so a quick mod of the script means you don't need wget:

#!/bin/sh
cd /tmp
curl -C - -O http://www.shallalist.de/Downloads/shallalist.tar.gz
tar -xvzf shallalist.tar.gz
cp -r BL/* /etc/dansguardian/blacklists/
rm -r /etc/dansguardian/blacklists/COPYRIGHT
rm -r /etc/dansguardian/blacklists/global_usage
date > /etc/dansguardian/blacklists/updated
/etc/init.d/dansguardian restart
exit 0

My change to the cgi file is below, it puts the update date in the box title.

In /home/httpd/cgi-bin/dansguardian.cgi find:

openbox('100%', 'left', _('URL Blacklist: Filter pages known to have content of the following categories.'), "blacklist");

Replace with this:
$output = `cat  /etc/dansguardian/blacklists/updated`;
$output = &cleanhtml($output,"y");
openbox('100%', 'left', _("URL Blacklist: Filter pages known to have content of the following categories. Update Date: $output"), "blacklist");

Notice the change from single to double quotes around the string.
Logged
entourage
Full Member
***
Offline Offline

Posts: 48


« Reply #19 on: Monday 17 August 2009, 11:56:49 pm »

I just got around to adding this to my Endian 2.2 install and I can confirm it works GREAT!!! 

Thanks for the effort!

BTW, if you're a business, don't forget to sign the Shallalist agreement.  It's free to use and they were very good about getting the info to me.
Logged
Dougie
Jr. Member
*
Offline Offline

Posts: 1


« Reply #20 on: Tuesday 18 August 2009, 08:41:48 pm »

Thanks worked for me. Community 2.2.
Logged
whoiam55
Full Member
***
Offline Offline

Posts: 71



WWW
« Reply #21 on: Tuesday 30 March 2010, 11:34:10 pm »

Today while reading up on urlblacklist.com I stuble upon this
Quote
Q 10. What do you mean about needing unsorting with DansGuardian?
The blacklist is provided as a gzipped tar file with directories containing text file lists. These lists are now provided in a sorted ordered list. DansGuardian, when reading in the files, will in memory sort the file. It needs to sort them to be able to quickly search them when doing the filtering. However DansGuardian uses the Quick Sort algorithm which is very quick to sort random lists but it will take a very LONG time to sort a list that is already sorted. DansGuardian will eventually start. If it does not then you might have a very old version with a bug. The input URL list being sorted should not affect any script or program and stop it actually working.

Q 11. How do I unsort the blacklist?
Unsorting a single file is very easy. Simply use the 'rl' or 'unsort' unix command on each of the text files. You can find information on these here:
http://ch.tudelft.nl/~arthur/rl/
http://www.vanheusden.com/unsort/

To unsort all the files in the blacklist in one go you could:
cd blacklistdir; find . -type f -exec rl '{}' -o '{}'.tmp \; -exec mv -f '{}'.tmp '{}' \;
This will find all the files in blacklistdir and subdirectories and randomize the lines. It should only take a few seconds if that. If you are using unsort instead of rl you will have to edit the above.

Pretty straight forward I guess.
Logged

सत्यमेव जयते!
g.vecchi
Full Member
***
Offline Offline

Posts: 21


« Reply #22 on: Friday 10 June 2011, 02:11:46 am »

Hi all!
I'm new of EndianFW and I'm trying to create a better blacklist for Content Filtering.

I'm running EFW Community 2.4.1 and I've modified jaebird script in this way:

Code:
#!/bin/sh
cd /tmp

if [ -a /tmp/blacklists ]
  then
    rm -r /tmp/blacklists
fi

curl "url_to_blacklist" -o /tmp/blacklists.tar.gz
tar -xvzf /tmp/blacklists.tar.gz
mv /etc/dansguardian/blacklists /etc/dansguardian/blacklists.bak
rm /tmp/blacklists/CATEGORIES
mkdir /etc/dansguardian/blacklists
cp -r /tmp/blacklists/* /etc/dansguardian/blacklists/


# Check categories file exists
if [ -f /tmp/blacklists/categories ]; then
  rm /tmp/blacklists/categories
else
  touch /tmp/blacklists/categories
fi

# Create categories file
for i in `ls /etc/dansguardian/blacklists/`; do
  if [ "$i" != "categories" ]; then
    echo "$i"="$i" >> /tmp/blacklists/categories;
  fi
done

cp /tmp/blacklists/categories /etc/dansguardian/blacklists/
rm -r /tmp/blacklists*
chmod 755 -R /etc/dansguardian/blacklists
chmod 644 /etc/dansguardian/blacklists/categories
/etc/init.d/dansguardian restart
exit 0

url_to_blacklist = bigblacklist url  <--- sorry for that but forum doen't allow me to post clickable link



but dansguardian failed to start because it says that

/etc/dansguardian/blacklists/violence/expressions

doesn't exist

So, I've modified /etc/dansguardian/profiles/1/bannedregexpurllist removing line about /etc/dansguardian/blacklists/violence/expressions and dansguardian restart to work: is that a good idea???

Thanks to all!
Logged
portucale
Jr. Member
*
Offline Offline

Posts: 1


« Reply #23 on: Tuesday 04 October 2011, 10:19:43 pm »

thanks to g.vecchi and addicional info i've read in other foruns, my script looks like this:

(create the file update_blacklist.sh and save it in /etc/cron.weekly/)

Code:
#!/bin/sh

cd /tmp

if [ -a /tmp/shallalist.tar.gz ]
  then
    rm -r /tmp/shallalist.tar.gz
fi

curl -C - -O SITE

if [ -a /tmp/BL ]
  then
    rm -r /tmp/BL
fi

tar -xvzf shallalist.tar.gz

if [ -a /etc/dansguardian/blacklists.bak ]
  then
    rm -r /etc/dansguardian/blacklists.bak
fi

mv /etc/dansguardian/blacklists /etc/dansguardian/blacklists.bak

mkdir /etc/dansguardian/blacklists/

cp -r /tmp/BL/* /etc/dansguardian/blacklists/

rm -r /etc/dansguardian/blacklists/COPYRIGHT

rm -r /etc/dansguardian/blacklists/global_usage

date > /etc/dansguardian/blacklists/updated

# Create categories file
for i in `ls /etc/dansguardian/blacklists/`; do
  if [ "$i" != "categories" ]; then
    echo "$i"="$i" >> /etc/dansguardian/blacklists/categories;
  fi
done

/etc/init.d/dansguardian restart

if [ -a /tmp/shallalist.tar.gz ]
  then
    rm -r /tmp/shallalist.tar.gz
fi

if [ -a /tmp/BL ]
  then
    rm -r /tmp/BL
fi

exit 0

IMPORTANT

replace SITE in the line
curl -C - -O SITE
with a blacklist file like shallalist.tar.gz

then in the file /etc/dansguardian/profiles/1/bannedregexpurllist remove the line with /etc/dansguardian/blacklists/violence/expressions
save the file and in shell do
(assuming the file you've create the file update_blacklist.sh in /etc/cron.weekly/

Code:
cd /etc/cron.weekly/
chmod 755 update_blacklist.sh
./update_blacklist.sh

this will execute the script
go to endian->proxy->contentfilter->edit
and in the tab Filter pages known to have content of the following categories. (URL Blacklist)
select the categories you want to block

after that you can restart dansguardian doing in shell

Code:
/etc/init.d/dansguardian restart

all should all get's ok!
Logged
Pages: 1 [2] Go Up Print 
« previous next »
Jump to:  

Page created in 0.172 seconds with 18 queries.
Powered by SMF 1.1 RC2 | SMF © 2001-2005, Lewis Media Design by 7dana.com