Debugging or Checking Out a Proxy

Before you trust your data using that nice, new shiny proxy that you found online. You’re going to want to check it out – so what can you do. Well believe it or not every single one of us has the perfect tool on our computer – it’s called telnet. Now you may think this is a little bit basic but you can actually get quite a lot of information on a proxy server just by using this simple program.

HTTP (Hyper Text Transfer Protocol) is the mainstay of our proxy, it’s raison d’etre if you like. Fortunately for us HTTP is a completely ASCII protocol operating in clear text which makes it perfect for using Telnet with. None of that complicated decompiling of binary data for us, all our responses can be read in plain (well a little Geeky) English.


Understanding Proxies

So How Do We Use Telnet to Debug?

It’s actually quite straight forward and uses the standard Telnet Syntax –

Telnet {Proxy Address} (Proxy Port}

So if you wanted to check out your college proxy server then simply –

Telnet 8080

This will get the telnet program to attempt to connect to the proxy server (or in fact any web server as well).   If you don’t get blocked by a firewall or restricted by policy you’ll get something like this –

Connected to

Escape character is   ‘^]’

Followed by a cursor sign (usually an underscore _).  When you’re at this point anything you’ll type will be sent to the server.

So here you can forward any HTTP requests directly to the server without using a browser.  But it will also allow you to see proper error codes and the responses the server is making.

For instance if you get the response

– telnet: Unable to connect to remote host: Connection refused

This suggests that the server process is not running or it’s not listening on the port you specified (telnet will connect by default on 23 if  you don’t specify).  It’s really great way of troubleshooting issues with web servers, proxies or any web enabled device.  It’s also helpful in determining when problems are occurring in other services,  for instance you can check out problems with Smart DNS  or HTTP services by logging on to their specific ports.


Type of Filtering and Ninja Bypassing

Internet filtering used to be relatively scarce but it’s extremely common now and takes a variety of forms.  The two most basic forms are URL and content filtering .

URL Filtering

Typical examples of URL filtering is where the requested URL of a web site is intercepted by the proxy or firewall and compared to a big list of ‘bad urls’.  If the URLs match then the request is denied and blocked.  In  this case the user is normally redirected to an error page, although in some cases the request will be logged and an administrator alerted.   It’s not a great system as if you have an extensive list of URLs it can have a big performance impact – and remember this impact is for all requests even those that don’t contain a blocked site.

In recent years some performance improvements have been made to alleviate the issues.  For instance some URL filtering systems use hash values of the URLs rather than the addresses themselves.  The hash values can be ordered so that the system can locate information faster (by jumping to specific points in the list rather than searching from start to finish).   Most systems you’ll find in corporate environments will use URL filtering to some extent.

There can be lots of other problems with filtering simply based on a list especially if you use the hash value searching system.  The URLs have to complete and only that exact, specific address is restricted.   Many websites have multiple domain names and aliases so any list has to have all these URLs listed too.

Content Filtering

Just like URL filtering has a noticeable impact on performance, the same can be said of content filtering.   Content filters look inside the data being transmitted – their goal is not only to block access to inappropriate sites but also to check for security risks.  A content filtering system will often be set to filter out specific objects like Java or ActiveX.   They also check for viruses and other security problems entering the network.

These filtering systems are very sophisticated – analysing the actual packet data though is bound to have an impact on any networks performance.  Content filters will usually defeat the use of anonymous proxies as the end URL is irrelevant – the data itself is being scanned which will reveal both the proxy address and the destination URL.   An example of one of the most widely used content filters is WebSense – which uses a variety of plug ins and runs on dedicated hardware strategically placed with a tap into all network traffic.

Ninja Bypassing of Filtering Systems

To defeat the URL filtering system is normally fairly straight forward, most anonymous ninja proxy servers available on the internet will suffice.  The only difficulty is that most URL lists contain a large selection of these sites – so if the one you use is on the list you’re going to get blocked.   Not only that but the administrator will likely be informed that someone is deliberately trying to bypass corporate restrictions.  If you set up your own using a hosting account and a Glype installation then you’ll likely be able to surf under the radar.

Unfortunately the mass majority of filtering devices now use both URL and Content filtering technology. The normal web proxy sites you’ll see on the internet promising you complete anonymity and the ability to bypass filters are completely useless. The content filter will look into the packet itself – the fact you are using a proxy and a fake ip are irrelevant.

There is only one effective way to defeat a genuine content filter and that is to encrypt your surfing. In this case the URLs and sites you are visiting are unable to be read by the content filters.