Category: filtering

Internet Filtering, Censorship, Surveillance and Stuff Like That

One of the many justifications used across the world by agencies, governments, regimes etc for spying on us and filtering internet feeds is that it actually protects us.    By that they generally mean by employing these tactics they are able to catch more terrorists, paedophiles and various nasty people using the internet for their naughtiness.  In fact in many sectors of society if you argue that the internet shouldn’t be monitored or filtered then you will often find yourself grouped with these unsavory characters. Now just to clarify I’m not talking about carefully targeted surveillance and filtering on suspects (fair enough on that) but the general broad monitoring and filtering on an entire population on the off chance of picking up something interesting !

The problem is that it’s utter rubbish for one very good reason – it simply doesn’t work.    It’s all very well a Government thinking that they can routinely pick up terrorists by swooping on a Facebook page – but in reality what sort of hardened operatives are they going to pick up?   One thing for sure they won’t be very clever – in fact you’ll probably pick up the likes of these two harmless muppets who tried to organise a riot on Facebook.   Their riot attracted no rioters and they were picked up and sentenced to four years (which will probably be reduced to 2 weeks on appeal).

Jordan Blackshaw, left, and Perry Sutcliffe-Keenan

Now to be honest I don’t know about you, but I might be prepared to concede a large part of my liberty and privacy if I thought the world would become a genuinely safer and better place.   However picking up the likes of these two hardly meets that criteria.

The point I’m trying to make is that when internet filtering, censoring and surveillance techniques are utilised the only people who are affected are those with nothing to hide, plus perhaps a few thick criminals/terrorists who are probably of limited danger.   There are many ways to circumvent filters, there are lots of ways to communicate anonymously and all those who need to are doing just that.

Do Al Qaeda communicate through Facebook, My Space or Twitter – I suspect not.  Do they send out their orders by emails in clear text with PDF attachments detailing their targets – of course they don’t.    They’ll be using TOR, encrypted emails, hidden web sites and communication networks on the Dark web.   There will be codes, ciphers and carefully devised communication methods and strategies plus loads of other stuff on here The Ninja Proxy!

Of course they might be like this lot from the rather funny film Four Lions –


But I suspect not.

Type of Filtering and Ninja Bypassing

Internet filtering used to be relatively scarce but it’s extremely common now and takes a variety of forms.  The two most basic forms are URL and content filtering .

URL Filtering

Typical examples of URL filtering is where the requested URL of a web site is intercepted by the proxy or firewall and compared to a big list of ‘bad urls’.  If the URLs match then the request is denied and blocked.  In  this case the user is normally redirected to an error page, although in some cases the request will be logged and an administrator alerted.   It’s not a great system as if you have an extensive list of URLs it can have a big performance impact – and remember this impact is for all requests even those that don’t contain a blocked site.

In recent years some performance improvements have been made to alleviate the issues.  For instance some URL filtering systems use hash values of the URLs rather than the addresses themselves.  The hash values can be ordered so that the system can locate information faster (by jumping to specific points in the list rather than searching from start to finish).   Most systems you’ll find in corporate environments will use URL filtering to some extent.

There can be lots of other problems with filtering simply based on a list especially if you use the hash value searching system.  The URLs have to complete and only that exact, specific address is restricted.   Many websites have multiple domain names and aliases so any list has to have all these URLs listed too.

Content Filtering

Just like URL filtering has a noticeable impact on performance, the same can be said of content filtering.   Content filters look inside the data being transmitted – their goal is not only to block access to inappropriate sites but also to check for security risks.  A content filtering system will often be set to filter out specific objects like Java or ActiveX.   They also check for viruses and other security problems entering the network.

These filtering systems are very sophisticated – analysing the actual packet data though is bound to have an impact on any networks performance.  Content filters will usually defeat the use of anonymous proxies as the end URL is irrelevant – the data itself is being scanned which will reveal both the proxy address and the destination URL.   An example of one of the most widely used content filters is WebSense – which uses a variety of plug ins and runs on dedicated hardware strategically placed with a tap into all network traffic.

Ninja Bypassing of Filtering Systems

To defeat the URL filtering system is normally fairly straight forward, most anonymous ninja proxy servers available on the internet will suffice.  The only difficulty is that most URL lists contain a large selection of these sites – so if the one you use is on the list you’re going to get blocked.   Not only that but the administrator will likely be informed that someone is deliberately trying to bypass corporate restrictions.  If you set up your own using a hosting account and a Glype installation then you’ll likely be able to surf under the radar.

Unfortunately the mass majority of filtering devices now use both URL and Content filtering technology. The normal web proxy sites you’ll see on the internet promising you complete anonymity and the ability to bypass filters are completely useless. The content filter will look into the packet itself – the fact you are using a proxy and a fake ip are irrelevant.

There is only one effective way to defeat a genuine content filter and that is to encrypt your surfing. In this case the URLs and sites you are visiting are unable to be read by the content filters.