You are currently browsing the Ninja Proxy Surfing posts tagged: proxies


Iran Prepares Itself for an Intranet

When countries start to heavily censor  the intranet,  it’s easy to imagine where  they’ll end up – running a State controlled intranet.  We can see it happening now in Iran, there’s news across the net reporting that the country is building up to it’s (ahem) democratic elections in June.   One of the steps they are taking is attempting to block all proxies and VPNs being used in the country.  It’s something China have been doing for years and although they have a much  more sophisticated approach it’s incredibly difficult to do – read here about the Chinese TOR probe.   Iran are reportedly trying to block all ‘non-approved’  VPNs and proxies basically to ensure that nobody is using them to avoid the countries growing content filters and blocked web site list.

Internet Access in Iran

Iranians will still be able to use the approved VPN providers, although why those who are  concerned with state spying and internet filtering would want to use these is completely irrational.   The reality is that it is an information war that the Iranians will lose, for every block or control they put up someone, somewhere will figure out a way around it.   For example you can block access to web sites and indeed VPN services in a variety of ways.

Create Blacklist of  Proxies and VPN Services 

This is what a lot of countries like Iran do initially and how many commercial filters work.  You just build up a database of specific IP addresses and URLs of known services and just completely block access to them.  This means that the user will not be able to make that initial connection to encrypt or bypass the content filters.  But there is a huge flaw with this technique, anyone with a little knowledge could set up a VPN or proxy service on a hosted server somewhere in minutes.  There are customised scripts and simple installations of proxies like Glype and Squid that can be set up by anyone on a shared server.  It’s simply impossible to keep track of all these servers – remember it’s the Iranian Regime’s IT workers VS the rest of the Internet – who’s your money on?

Deep Packet and Pattern Inspection

You can attempt a more sophisticated technique by trying to look inside the traffic and figure out when a VPN or proxy is being used.  Even if you control the internet boundaries in your country this is very difficult to do.  For a start it’s almost impossible to analyse every packet that leaves and enters the country via the internet.  The amount of resources you will use would be enormous, not to say you end up pretty much crippling internet access at the same time.  So you have to restrict your checks to certain patterns – perhaps selecting traffic leaving or using specific ports – maybe 443 for example.  This is still going to use an enormous amount of resources and of course there’s nothing to say a specific service has to use a specific port number for connection.

Just look at one of the configuration screens of my preferred security software – Identity Cloaker.

identitycloakerconfiguration

Just look at the options there in one screen for cloaking, modifying port redirection and simply changing individual elements of the connection protocol.   It’s extremely difficult to look for specific patterns when there is this amount of customization is available in the connection methods.  Of course most  security/VPN software don’t  offer anywhere near this level of sophistication, but the market would soon be created if there is a demand created by increased filtering.

Wrecking the Digital Economy

This might not be of concern to Iran, but for countries like China it is a very real issue.  Whether they like it or not any successful business needs the internet, if you start breaking or restricting the infrastructure they’ll simply go elsewhere.  Any multinational business will use VPNs to connect back to their corporate networks safely and securely.  Will they be prepared to use Iranian approved VPNs instead of the tried and tested commercial alternatives?  Every web site that is blocked, every VPN closed down makes it more and more difficult to operate in a specific country.  The benefits of a digital economy are eroded and a countries economy will without doubt suffer.  As mentioned it may not matter if religious and political ideals are the primary goal, but as we have seen from the Arab spring – economic woes causes revolutions too.

These are just a few simple reasons why many believe that the technological reasons mean that inevitably the level of control required by someone like the Iranian Government will lead to an intranet.  Just to clarify  that would involve blocking all access to the outside and internet and restricting access to content created and hosted in Iran.  Sounds fun, doesn’t it but Iran has been working on this since the Spring of 2011 and is the only way they can control what people see from their phones and laptops.  Of course they’ll be dragging the country back into the dark ages when they do it but perhaps that’s not a problem.

National Governments that Censor the Internet

According to Wikipedia, the term “internet censorship” is defined as “the control or suppression of the publishing of, or access to, information on the Internet.” Internet censorship is implemented by national governments or private organizations – delegated by governmental influences – for several reasons, with emphasis on: religion, moral issues and unlawful business schemes.

Enemies of the Internet List

Reporters without Borders (or “RWB”) is a French, non-profit organization that advocates freedom of information and press. This organization has compiled a list of countries that are deemed “Internet enemies” due to their methods of cyber censorship. The flagged nations are: Armenia, Bahrain, Belarus, Burma, People’s Republic of China, Cuba, Iran, North Korea, Saudi Arabia, Syria, Turkmenistan, Uzbekistan and Vietnam.

In 2009, Belarus was added to the “Enemies of the Internet” list, was subsequently removed, and was added again in 2012. In 2011, Egypt was added to the “Enemies of the Internet” list. In 2012, Tunisia was added to the list after being removed in 2011. Egypt was also removed in 2011 and was added again in 2012. The small kingdom of Bahrain was also added to the list in 2012.

Countries under Surveillance

The RWB further compiled an “Under Surveillance” list. Nations under this category are considered to give cause for concern about the possibility of increased Internet censorship. The current list spans the following countries: Australia, Egypt, Eritrea, France, India, Kazakhstan, Malaysia, Russia, South Korea, Sri Lanka, Thailand, Tunisia, Turkey and the United Arab Emirates.

The “Countries under Surveillance” list, introduced in 2008, listed 10 nations which used surveillance on users’ Internet activities or otherwise impeded people’s rights, without blocking massive amounts of information. Between 2008 and 2012 the number of countries listed grew to 16 but subsequently fell to 14. Jordan in 2009, Tajikistan in 2009, and Yemen in 2010 were dropped from the list.

Australia in 2009, France in 2011, Russia in 2010, South Korea in 2009, Turkey in 2010 were added.

Bahrain, Eritrea, Malaysia, and Sri Lanka dropped from the list in 2010, but were added again in
2011. Libya dropped from the list in 2009, added again in 2011, and then dropped in 2012. Venezuela was added in 2011 and then dropped in 2012.

Internet Freedom Setbacks

Azerbaijan, Libya, Malaysia, Pakistan, Rwanda, Russia, and Sri Lanka are seven nations that are at particular risk of suffering Internet freedom setbacks in 2013.  These nations are relatively free of government interference and censorship on the Internet for their citizens; however, they also maintain separate governments either known to be: a) repressive of freedom of the press in traditional forms of media, or b) they have recently introduced laws that significantly affect online freedoms of expression in negative ways for their citizens.

There are two ways to address internet censorship. The first is to change an IP address to an address under a non-censored nation.  The second is to access a search engine website that acts as a host by displaying all results through their site. Governmental censorship reduces freedom of expression, while simultaneously revoking basic, human rights. For more information on this cyber suppressive trend, refer to the Internet Censorship infographic found below.

internet censors

 

Source: http://en.wikipedia.org/wiki/Internet_censorship

Source: http://en.wikipedia.org/wiki/Internet_censorship_by_country
Source: http://march12.rsf.org/i/Report_EnemiesoftheInternet_2012.pdf

Video Not Available in Your Country

Ever seen this message, your mate has just sent you a message something like – “you have to watch this, I lolzed so much I nearly died. Lol lol”. So you make a mental note to get some cleverer mates, then click on his link and are presented with a YouTube message like this -

It might not be that specific one, it might be the – Video is not available in your country, or this video is not available – pretty annoying huh!

There are a variety of reasons for these messages but the vast majority of them are due to your IP address or more specifically the location of your IP address. Some are blocked automatically (like lots of BBC content outside the UK), some blocked by Youtube and some like this example specifically targeted by the person who uploaded it. So let’s take this video as an example - http://www.youtube.com/watch?v=gYYUdDNJXaw&feature=related
the uploader appears to be from Germany

It’s likely that this video relates to some some of movie, music or TV show that is only licensed for free in Germany. So when YouTube checks our location we will be blocked (I’m in the UK at the moment). Anyway what we are going to do is use Identity Cloaker to connect via a German proxy server and then try and view the video again.

So when we start YouTube it will now think we are in Germany and so apply restrictions based on this country.   Now we can see the video and don’t get the – Video Not Available in your Country error message.  Sorry it’s so small – graphic size was huge when I uploaded, click on it to make it a bit bigger.

It’s actually a movie broadcast by the German TV company Kanal which is why it’s been uploaded with the restriction of being seen only in Germany.  Remember though when you are using a German proxy server, you may get blocked from other stuff which you can normally see without problems.  It’s a pain but I’m afraid you need to switch servers depending on where the site is based – the most essential to have are firstly your home country and then probably fast UK IP Address, US and a good European server in somewhere like France or Germany.

So if you want to say goodbye to all these Video Not Available messages invest in a proxy service or hunt down some free ones online if you have time.

What is a Web Proxy Server?

A long time ago, at least in the context of the Internet – we used to call proxy servers –  gateways. In  fact the first WWW gateway was created at CERN by the World Wide Web team led by Tim Berners-Lee.  Yep that BernersLee – the man who invented the World Wide Web !

So how can we define these gateways, what do they actually do?  Well the most common description is that they are devices which forward packets between different networks.  Of course sometimes these networks are fairly different so the gateways need to translate protocols before they forward them on.  The difficulty here was that two distinct types of devices were being grouped together under the title – gateways – which needed defining.

The first type were Internet gateways which acted both as a firewall and a gateway to the internet.  These would sit in front of secure private networks and allow access both inbound and outbound – these were defined as proxy servers.  The other type were information gateways which usually acted on behalf of a server rather than the client.  These were defined as ‘gateways’ although some call them reverse proxies just to keep it slightly confusing.

So there are even quite a few different type of proxy servers, ranging from the one page web proxies you’ll find on loads of web sites, to big corporate proxy servers which you use to access the internet from work or college.

There are however some common properties that all proxies should share -

First of all they should be transparent.  That is to say they should not affect the end result, the client should receive exactly the same result from the web site whether you use a proxy or not.

Second, the decision to use a proxy should be instigated and controlled from the client.  Although in most corporate networks this is true – the use of a proxy is normally hard coded in to the configuration.  Most Windows clients will enforce the browser to surf through the corporate network.  Next time at work or school if you look in your browser under connections you’ll probably see a proxy server address set here which you cannot alter. (Under IE you’ll find it under Internet Options/connections/LAN settings).

Finally the last main property is that the destination server or web site should be completely unaffected by the use of a proxy server.

These three definitions are however slightly under threat particularly because of the increasing use of geolocation.  This technology targets and controls content based on the geographical location of the client. However if you surf through a proxy server, then it is the location of  the proxy that determines what you can access or see.  This is why proxies are becoming so popular – if you have access to the right proxies you can access any content you like.  For instance if you want to access BBC Iplayer and you live outside the UK you’ll have your access blocked.  However if  you connect through a UK proxy server, then you’ll be considered a UK surfer and be allowed to use it.

Why Can’t I Use a Proxy

We’ve all been there – you’re stuck in work or school, and frankly bored out of your brain.   Sure you have internet access but all the most interesting sites are blocked -

  • Facebook Blocked
  • Youtube Blocked
  • MySpace Blocked
  • World of Warcraft (games and forum) Blocked

So why’s it happening and what can you do about it?

Your company or school controls your access to the internet at several points and is blocking your access at several levels.

The first control is probably through their own proxy server.  If you go and look in Tools/Internet Options/Connections/LAN Settings or  something like that in different browsers you’ll probably see a proxy server set.  That address will be a server controlled by your company where they force all internet traffic.  If they’ve done a decent job you won’t be able to change this.

The settings will normally be deployed by something called GPO (group Policy Objects) which are the way most organisations control what their computer looks like.  These apply settings like specific desktops, screensavers, Internet Explorer settings each time you boot up your computer.

Therefore absolutely everything you request goes through the company proxy server.  You might think you’re being clever searching for ninja proxy sites on the internet but I’m afraid you’re not.  All you are doing is creating a log of you searching for ‘ninja proxy sites online’, and letting administrators know you want to bypass their settings. The proxy server will be set to filter out all such requests by a variety of methods.  The most common one will be a huge list of URLs containing all the dodgy one page, Glype proxy installations online.

So you need to bypass this proxy server or do you?

If the organisation has their network set up properly then even by using an alternative browser or modifying the proxy settings in IE will not work anyway.  The reason is that your company firewall, the hardware device which controls all the traffic in and out of your network should only allow web traffic out from one specific address – the proxy server.   So if you bypass this your request will come from your specific IP address and get blocked.

Then a couple of things might happen -

  • The alert will be flagged on the firewall (Web requests from an incorrect internal client)
  • The administrator will track down the PC and find out it’s been modified.

But don’t worry in reality probably nobody ever looks at  the logs and most firewalls generate so many alerts that nobody ever looks at those either.

The point is your searching for online web proxies is simply a waste of time.  To bypass most corporate proxies you need to go through that proxy and not around it.  Through it because any other originating IP address will get blocked and may possibly  wake up your IT Department.  But you need to stop the proxy blocking access based on the content (what you are requesting) and the URL (the actual site you want to visit).

There are two things you can do to allow this – first you need encryption so that nothing can see inside your web request and secondly you need some low key server outside the network to relay your request.  These two requirements if implemented correctly will allow you to tunnel through any corporate network firewall or proxy and also keep your surfing private from the administrators and logs.

Debugging or Checking Out a Proxy

Before you trust your data using that nice, new shiny proxy that you found online. You’re going to want to check it out – so what can you do. Well believe it or not every single one of us has the perfect tool on our computer – it’s called telnet. Now you may think this is a little bit basic but you can actually get quite a lot of information on a proxy server just by using this simple program.

HTTP (Hyper Text Transfer Protocol) is the mainstay of our proxy, it’s raison d’etre if you like. Fortunately for us HTTP is a completely ASCII protocol operating in clear text which makes it perfect for using Telnet with. None of that complicated decompiling of binary data for us, all our responses can be read in plain (well a little Geeky) English.

 

Understanding Proxies

So How Do We Use Telnet to Debug?

It’s actually quite straight forward and uses the standard Telnet Syntax -

Telnet {Proxy Address} (Proxy Port}

So if you wanted to check out your college proxy server then simply -

Telnet collegeproxy.com 8080

This will get the telnet program to attempt to connect to the proxy server (or in fact any web server as well).   If you don’t get blocked by a firewall or restricted by policy you’ll get something like this -

Connected to collegeproxy.com

Escape character is   ‘^]’

Followed by a cursor sign (usually an underscore _).  When you’re at this point anything you’ll type will be sent to the server.

So here you can forward any HTTP requests directly to the server without using a browser.  But it will also allow you to see proper error codes and the responses the server is making.

For instance if you get the response

- telnet: Unable to connect to remote host: Connection refused

This suggests that the server process is not running or it’s not listening on the port you specified (telnet will connect by default on 23 if  you don’t specify).  It’s really great way of troubleshooting issues with web servers, proxies or any web enabled device.