Category: content filtering

Residential IP Provider

We talk a lot about proxies in this blog, how to use them to stay secure, protecting our anonymity and really important stuff like being able to watch the BBC on our holidays.    In truth, as far as watching online media sites go proxies are very little use anymore because they’re very easy to detect.   If you want to watch a multimedia site online then you’re normally going to need a VPN as they are much more difficult to detect.

There is another side of the blocking war technology, which doesn’t involve the type of server you’re connecting to but rather the IP address it is assigned.  You see there’s another crucial distinction as far as the network address you use on these servers – commercial or residential.  Virtually all IP addresses that the ordinary individual can get access to falls under these two categories.   So it’s worth just briefly clarifying what we mean by these:

  • Commercial IP Addresses – these are assigned to public and private companies and commercial datacentres.  Indeed 99% of proxy and VPN services are hosted in commercial datacentres which is why they’re allocated commercial addresses.
  • Residential IP Addresses – if you simply connect to the internet through your ISP like most people, then you’ll have a residential address.  These are only allocated to home users via their normal internet connections whether they’re broadband, cable or even dial up.

It seems a minor difference, after all they’re both just standard IP addresses but the distinction is becoming more crucial.   In 2016 Netflix became the first ‘media giant’ to restrict access to their content based on the classification of your IP address.  It’s been used before in smaller organisations and in other sectors but this was a major shift in region locking.  If you sat on a company network and wanted to watch Netflix using your computer you’d be blocked, didn’t even matter if you owned the company!

After years of trying to enforce Netflix users to stick to the locale they were based in, they finally decided that only those who originated from a residential IP provider would have access.  This worked big time, indeed compared to the years of blocking proxies, checking for concurrent connections and trying to detect VPNs it was incredibly effective.  The reason was simple all the circumvention tools like proxies and VPNs had one thing in common, they all had commercial IP addresses assigned from the datacentres where there servers sat.

People Suddenly Need a Residential IP Provider

There was some collateral damage, but basically it made Netflix only accessible from home connections.  More importantly users could not use VPNs and proxies to switch to a better version of Netflix, the US version has more than double the content of many regions.

In fact what Netflix was doing wasn’t that original, in other areas commercial websites have been using this classification for years. Anyone with even a rudimentary knowledge of programming can use incoming IP address information to determine both the region and classification of the IP address. Using the classification is an easy way to ensure all the inbound connections are from legitimate residential users.  They can also determine specific countries and regions from the IP address to – all crucial information if you’re running any sort of e-commerce site too!

Most online resources and commercial sites are directed at home based customers which is why having a residential IP address is crucial, residential traffic has certain characteristics.  Normally that’s fine but there are also many situations where you need more than one address and that’s extremely difficult to do.  For example people involved in online marketing or research need the ability to process numerous requests from social media sites, advertisers, search engines and even online retailers.  Unfortunately if you try and do this from a single solitary internet connection and address you’ll get blocked pretty quickly.

residential IP provider

Whether it’s the classification, origin or more likely the fact it’s being used for repeated requests – it’s pretty simple to get blocked by a web site nowadays.

People who use marketing and research tools will get blocked very quickly for example as the multiple requests are logged. Even posting stuff to social media using applications can have this effect very quickly too.

Others use tools in order to buy multiple copies of items or place lots of adverts. Again if you are stuck using your single default, residential IP address you’ll get an IP ban very quickly.  the more requests you or the application makes then the quicker it will happen.   Even something like a residential VPN won’t help because although the actual connection is encrypted, the mere fact that multiple concurrent connections are established from the same IP is easy to detect.

For example there are lots of people who use software to buy multiple items in order to sell them later.  There are computer programs designed to do this for you, making multiple requests and placing lots of items in shopping baskets automatically.

They are usually found in sectors which have limited or restricted availability like designer trainers/sneakers, concert and sporting tickets and similar items.  When items like these are sold online then the restrictions are normally enforced by the classification and location of the IP address.

Let’s consider buying concerts for a very popular rock band, the tickets always sell out extremely quickly.  How do you ensure that people don’t buy extra tickets to sell online at inflated prices?  Well imagine you’re sitting there looking at the requests come in for a Dallas based concert and you see hundreds of requests from IP addresses originating from New Delhi? These obviously don’t look like legitimate requests from fans and the website will almost certainly block or restrict access from these addresses.

There is a workaround method that can be used and is implemented very profitably by people online – they hide their locations by using home based IP addresses originating in ‘non-suspicious’ locations.  However to do this you need to buy residential IPs which are bought from a residential IP provider.  These are normally bought by buying access to dedicated residential proxies which have home based addresses assigned to them.

Some of these networks are extensive and extremely sophisticated indeed they need to be to fool the websites involved.  The most advanced are called residential backconnect proxies, which are much more than simply standard static residential proxies.   They are networks consisting of thousands of residential IP addresses all linked together through single backconnect proxy gateways.  These are much more difficult to detect as they offer the ability to rotate IP addresses automatically and also the referrer data that is sent from individual connections.   Any sort of automated bot which buys, researches or posts will need access to rotating residential proxies in order to function with any volume.

These networks are of course extremely difficult and expensive to set up, primarily because getting access to residential IP addresses is very difficult.  Many have taken years to set up, it’s not that hard to learn how to make residential proxies but it’s much,much harder to set one up!  It’s worth remembering this before you go searching to buy residential proxies, they are much more expensive to run than normal one.  Which is why the people running automated tools like ‘sneaker bots’ or ‘ticket bots’ are always searching for reliable sources, you’ll even see them referred to as residential sneaker proxies as they’re often used with these bots to buy lots of sneakers from large online retailers like Nike, Adidas and Footlocker.

There’s only two or there of these with large enough residential networks to make most of these bots and tools work properly.  Here’s one of the oldest and most respected networks – they’re called Storm Proxies.   They’re unique in that they’re the only ones who actually own the hardware and addresses (the registrations) unlike their competitors.  Indeed many of the other providers have little direct control over their residential IP address inventory, e.g. Illuminati who’s network consists of piggybacking off the connections of a free VPN called Hola.

It’s definitely the fastest, safest and best residential IP provider and you can try out their proxies here

Storm

Check out their residential proxy trial which you can test for 48 hours and get your money refunded if they don’t work for you.

Best Smart DNS vs VPN – (Making a Choice)

For the expat, the traveller or the film/TV buff the internet can often seem to contain lots of filters and blocks on their favourite websites.  This is because of something called geo-blocking or  geo-restrictions (also a host of of other names) and is effectively a system used to limit access to web content based on your physical location.

So for example if you try and watch coverage of the UK election madness on the BBC website from outside the UK the geo-restrictions will stop you watching. Try and access your US Netflix or Hulu account whilst travelling and the same thing will happen.  It seems that on the internet your  physical location shouldn’t really matter, however it does – very much.

Smart DNS vs VPN

In fact this practice is growing exponentially, literally thousands of websites restrict access based on your location.   Just browse on YouTube and you’ll find thousands of videos on that site which are restricted to specific countries.  It can be very frustrating, especially for those who travel a lot and inevitably people find one of two solutions to bypass these restrictions – Smart DNS or VPN.

The technology for these has been around for a long time, and both can be used to access most region blocked websites.  So what’s the difference, which one should you choose?

Should it be Smart DNS or a VPN Service?

Both of these technologies are mostly effective in bypassing most region blocks, however the way they work is quite different.

VPN (Virtual Private Network) – these are services which create secure, encrypted tunnels between your computer and a specially configured VPN server.  They have been used for decades to provide security by encrypting all your data and anonymity by hiding your IP address and location.  When you are connected to a VPN server the website cannot see you true location only that of the VPN server.

To bypass the region locks you just need to ensure that the VPN server is based in the location that does have access.  So for instance to access the BBC iPlayer you’d need a UK VPN server, a US one for Hulu and so on.   The connections are made from your device on demand, so you could connect to a UK VPN from your computer or tablet and watch the BBC or ITV then disconnect and browse normally.  the majority of the VPN services will offer servers based in many countries, so you can just select which one you need.

Smart DNS – is a newer technology based on using specially configured DNS (Domain Name Service) Servers.  These normally just look up the IP address or name of the server you are trying to access, but Smart DNS servers offer an additional service.   They are configured to intercept requests to certain region locked websites and route the connection through a server based in that particular country.

So if you request access to the Hulu website for example to watch a video, the Smart DNS server would automatically route your connection through a US based server.  This means that your connection is only rerouted through a different server when it’s required to bypass a region lock.

So that’s it really – both will usually allow access to region locked websites, so which one should you choose?

Well firstly the price – you’ll find that both are relatively inexpensive however Smart DNS will normally be slightly cheaper.  This is because a Smart DNS server routes through a simpler server and only incurs bandwidth charges for specific servers.   A VPN service will tunnel your entire connection through the VPN server and therefore the bandwidth costs will be much greater.

Smart DNS is also easier to set up on different network devices, it requires no software or client component.  You just change your DNS settings to point at  the Smart DNS servers, therefore it’s simpler for things like Smart TVs and media devices where you can’t always set up a VPN connection.   You can even set it up directly on your router fairly easily too, this has the advantage of applying the settings to all devices on your network.  It’s not always convenient to do this, however it is essential if you want to apply to devices which have no configurable network settings.  This video is a demonstration of setting up best Smart DNS directly on a router –


The VPN is the only one which provides security and anonymity.  Not only is all your data encrypted, your identity is hidden too.  You should always use a VPN to access secure sites like email, online banking and Paypal when using public internet access like Wifi hotspots.

The choice between smart dns vs VPN really depends on your circumstances if you have any requirements for privacy, security and encryption then a VPN is definitely your only option.  Remember Smart DNS services provide no security beyond basic routing to bypass region locks.  A VPN connection is also much harder to detect than a Smart DNS relay, so they generally work better with sites like Netflix which try and block the workarounds.

Here’s two options –

IDC is a full security product with a super fast VPN service  for accessing BBC Iplayer, Hulu and all media sites.  They have loads of  USA and UK based servers so if you want to watch the BBC Iplayer service then it’s probably your best option. They do have lots of servers in the France, Germany, Australia, Canada and throughout Europe as well though.  They also don’t automatically renew your subscription either which I like.

Overplay is another great little company, I like their VPN with Smart DNS  which is easy to use.  Lots of US servers included in the standard subscription. They also have the widest selection of servers although perhaps many won’t use most of them.  If you need a server in somewhere unusual they are most likely to have them.

Hey Look, That’s Suspicious. Hide IP all Internet Traffic

One very important aspect of privacy that is usually overlooked is that of discretion.  If you want to remain private, then it’s important not to stand out – in IT circles it’s often called security by obscurity.   I remember once having to investigate unauthorized use of a database system by an employee.   The scenario was that this particular application could be used to look up all sorts of personal details about people however this was strictly  forbidden.

When I started to investigate, it took about 2 minutes from looking at the logs to identifying the culprit.  The logs were huge, but all access was identified by the user access name in one column.  The usernames were all in a long format like this –  user/2347643-1964 all except one whose username was Bob21 (slightly changed to protect the stupid), and it stood out a mile.  The user had created his own account and used it to look up details of a love rival, but had failed to keep to the same username format as everyone else.   It yelled – check me out, very loudly indeed – he was discovered and subsequently lost his job.

hide vpn traffic

The point is that it’s all very well having a super secure and encrypted connection, however if this is too obvious you run the risk of making yourself a target and inviting investigation.  This is particularly relevant in using a proxy or VPN or proxy to hide yourself online.  Sure they work and a properly configured VPN over port 80 does keep you very secure, however if anyone looks at the logs the VPN user can stand out.

The reason is that in standard ISP logs a normal web user will have a variety of internet requests to all sorts of different web servers and IP addresses.  The VPN user will have all those hidden and will interact with the single IP address of the VPN server.  This stands out, the same IP addresses being connected almost permanently and no requests made to anywhere else.  The IP address is either a fascinating web site or more likely a proxy or VPN – it’s also simple to search and filter for this sort of behaviour.

Fortunately it is possible to hide VPN traffic by using certain highly secure VPN services which can be made almost invisible too with a few carefully configured options.   Here’s an example of those settings that you can configure in Identity Cloaker.

Basically you need to ensure that you rotate the IP address you access periodically – so that a variety of addresses appear. This suggest normal web browsing rather than a single encrypted connection which can help to hide openvpn traffic too. Other options are to allow the cloaking of different applications, so that some requests go direct to the site whilst others are redirected through the VPN tunnel.

After all if you’re streaming video from a recognised site, do you need encryption? Is it a secret?  You might already run openvpn over https too, and by making this sort of behaviour accessible and readable you control what aspects of your online activity is private and which isn’t.   These are simple tricks but extremely useful if you’re serious about tying to hide VPN traffic properly.

It’s not that difficult if you have these options in your VPN service to melt back into the crowd. However it makes a huge difference to the level of privacy you enjoy. A little obscurity is an essential element in keeping yourself safe online, any security adviser knows all too well that you should avoid making yourself into a target.

 

UK Users to Receive Piracy Warning Letters

A couple of weeks ago I sat in the front room of an elderly neighbour, on the TV screen was a Premier League football match being broadcast on Sky Sports.    I was surprised because I knew this chap struggled on a basic pension and the Sky Sports package is not cheap!   However I then noticed the digital box, it was not the standard Sky digital  box but a custom TV media box complete with VPNs.

For a few pounds a year to cover updates, this box was pre-installed with a version of Kodi and a few extras which granted free access to every single Sky subscription channel.  The cost of these channels if paid for legitimately would be over a hundred pounds a month, yet the cost for these was virtually nothing.  The box even simulated the Sky selection screen which meant that this 80 year old man was happily surfing at the cutting edge of digital piracy.

A New Breed of Digital Pirate?

It turns out his nephew has installed it and he’d been using it for several months.   He was blissfully unaware that technically he was stealing all this content from Sky and to be honest I didn’t feel the need to spoil his enjoyment by telling him

The reality is that in the UK and indeed across the world, digital piracy is starting to hit the mainstream.  In millions of households, neat little digital boxes sit happily under TVs streaming illegal copies of US cable channels or UK satellite channels.  It’s got to the point that it’s so common that most people don’t even consider it illegal, something like taping a radio programme or copying a DVD.  After all why pay a fortune monthly to some huge media conglomerate when you can purchase a pre-installed media streamer that supplies the same for nothing.

It does of course, cost the media companies huge amounts of lost revenue and obviously they are trying to stop this.  In the UK this month, will see the first phase of action designed to stop this behaviour.  The biggest ISPs in the UK will all be sending out emails to any individual who’s internet connection is being used to download copyrighted material illegally.

It’s been discussed for years but has always been postponed for a variety of reasons.   For example there was a lot controversy when copyright holders started using a practice dubbed as speculative invoicing.  These were basically demands for money threatening legal action against anyone who’s internet connection was being used to download copyrighted material – you can read about in this article – Bittorrents Monitored.   The issue has never been detecting the downloads but rather what actions can be taken, legally it was very difficult to prove an individual was responsible even if their internet connection was being used.

The letters will be sent in the form of emails, and will simply inform the user that their internet connection is being used to download copyrighted material and information about where it can be obtained legitimately.    There will be no threats, fines or further action and critics have pointed out that it will have little effect.   The action will only target P2P users, those who download using torrents and file shares – however the use of these methods has fallen dramatically over the last few years.  The majority of people who view copyrighted material now stream directly using these TV boxes and programs such as Kodi which are slightly more difficult to detect.

It is likely that these users will be targeted later although who knows how long this will take.

IP Address Mapping Hell in Kansas

Is there such a thing as a ‘digital hell’ well although it sounds like some sort of melodramatic media headline, one couple in Kansas could arguably have been living there for several years.

Everything that is connected to the internet has an IP address, every computer, laptop, tablet or smart phone needs some sort of address in order to communicate on the world wide web. Tracking, mapping and filtering these addresses is big business and many companies have sprung up providing accurate information on the IP address attached to your device.

Obviously knowing the location is one major part of the puzzle and there are several services for looking up the physical location of an IP address. You can have a look here at where your IP address appears to be located – https://www.whatismyip.com/ – did it return your correct location?  Sometimes these can be very accurate, the information sourced from companies like MaxMind has been built up over many years through a variety of methods. The information is used for a variety of reasons, from targeting advertising to region locking and filtering used by companies like Netflix

Sometimes, however this information is not very accurate at all  but sufficient if you just want a specific country or region. However when a company like MaxMind have no relevant data on an IP address they will tend to resort to assigning a default location. For example if they have no further information other than country is USA, Maxmind will return a default location – the geographic center of the United States.

ipad-632394_640-1

Sounds logical? It is until you realise that located in the geographical center of the US is a small farm in Kansas owned by James and Theresa Arnold. Furthermore there are quite a few IP addresses which are registered to this ‘default location’ – specifically just over 600 million addresses.

Now it might seem that this isn’t really a problem, but unfortunately this is not the case. These 600 million addresses are real and being used online all the time – and of course with such a huge volume some of these addresses are being used for all sorts of activities. Spammers, hackers, cyber crime, terrorists, pedophiles are all using these IP addresses online and when anyone tries to investigate their location – they are directed to this small rural farm in Kansas.

For years the couple have been subject to all sorts of accusations – they’ve had visits from law enforcement agencies, public officials, ordinary people who’ve been crime victims and have tracked the IP address back to the Arnold’s home address. You can imagine the volume when even a small percentage of 600 million addresses are used for criminal purposes.

It’s not the only situation like this, there is a house located at the end of a cul-de-sac in Ashburn, Virginia which has similar problems. The town itself is the home to several huge data centers and server farms, all with registered commercial IP addresses – the house was unfortunately given as the default location for millions more IP addresses with similar results – strange accusations and police raids being a common occurrence.

Fortunately there should be a happy ending for both these parties as the ‘default locations’ for unknown IP addresses is being changed to non-residential addresses such as the middle of a lake! The Arnold’s though are unsurprisingly also seeking some financial compensation for the distress and inconvenience over the year, and you can hardly blame them!