Too many security products trade on fear, uncertainty, and doubt among readers and the media. At the same time, giving a positive review to a flawed product risks putting people’s privacy and even their safety in danger. This is especially true for Virtual Private Networks, or VPNs, which are sometimes used to circumvent the censorship of oppressive regimes and are always used to secure web traffic. To review these products, we aim to balance many factors to produce reviews that are factual and useful to our readers.
Real-World Data
10Beasts prides itself on using hands-on experience and testing data to inform its reviews, but the expectations and concerns of readers are an important consideration, too. Accordingly, 10Beasts recently undertook a VPN survey of some 1,000 respondents, asking them about their knowledge of VPNs and what they expected from such products.
We were surprised to find that among our readers, 71 percent had never used a VPN. Despite that, we have traffic data to show that there is great interest among our readers regarding VPNs. Therefore, we focus our reviews on the basics of what a VPN should provide and strive, with every review, to educate our readers as well.
Additionally, 47 percent of respondents said they expected a VPN to be free. That’s a far cry from what we see in the actual VPN landscape, where the average price of a top-rated VPN service is around $10.70 per month. It’s clear, however, that our readers are interested in value and skeptical of expensive services, so pricing is an important consideration in our reviews.
Finally, the largest portion of our readers, at 25 percent, rightly identify internet service providers as the biggest threat to their online privacy. A close second picked Facebook as the biggest threat to privacy, at 24 percent. Google and the National Security Agency came in a close third and fourth, at 19 and 18 percent, respectively. Clearly, our readers understand that there are threats to their privacy. Our reviews must consider those threats and fears, but with the aim of informing our readers, rather than stoking paranoia among them.
Pricing and Plans
Keeping in mind our readers’ concern with price, when we review VPNs, we always opt for the monthly package and report that price in the reviews. We feel this gives the best apples-to-apples price comparisons. While nearly all VPN services will offer a discount when you sign up for a long term subscription, our goal is to report the base-level price for each service and not the discounts you might get for signing up for a year.
Another reason we choose the monthly plans is because we want to encourage readers to start with a short-term subscription for a VPN. Too often we have received emails from readers who spent $60 (or more) on a year-long subscription to a VPN only to discover they hate it. It’s far better to try out a service for a month or three, and decide to spring for a long-term, discounted billing plan only when you’re certain you want to keep it. Consider the extra money you spend up front to be a down payment against buyers’ remorse.
VPN Features
With more and more VPN services popping up, companies have begun adding more and varied features to their offerings in order to stand out. In 10Beasts’s review of each service, we strive to report as many features as we can but to focus on the ones we believe are the most significant and reflect the value of a service. The number of devices the VPN service allows you to connect simultaneously, for example, is a concrete measurement of value and a point we always report.
In our reviews we also note whether or not a service provides ad-blocking or malware blocking with their service, and if the VPN allows BitTorrent or other P2P services on its network. We do not test the efficacy of ad-blocking or malware blocking, partly because it is not part of a core VPN product but also because we feel stand-alone tools address those concerns better.
We also check to see if Netflix is accessible while connected to a US server on each VPN service. It’s not unusual for streaming services to block VPNs, as it allows them to enforce regional licensing agreements. Star Trek: Discovery, for example, is available on Netflix outside the US, but you’ll need a CBS All Access account to watch it in the US. Does this VPN work with Netflix? is a question that anyone who writes about VPNs hears all the time.
Each VPN review also notes the most significant add-ons available from a VPN service. These usually include static IP addresses, additional simultaneous connections, and so on. We do not test these additions.
Server Numbers and Distribution
Our VPN reviews include a current count of the number of servers provided by the VPN company. The number of servers can give you a rough sense of how robust a VPN service is. That’s because with more servers available, the company can connect fewer people to each server. That means a bigger slice of the bandwidth pie for each person assigned to a given server.
This figure, however, is just part of the story. Most VPN companies spin up new servers to meet demand as necessary, causing the precise number of server to change often. It also doesn’t make sense for a small company with only a few thousand subscribers to have as many servers available as a company with a million subscribers. We try to balance these considerations in our reviews.
In addition to the number of servers, we also look at how many different server locations are available and how widespread those locations are. We call this “geographic diversity,” and give preference to services with lots of servers in many different parts of the world. This is particularly important to frequent travelers and users overseas, since a VPN server closer to their computer will likely mean a faster and more reliable connection. For users in the US, more VPN server locations mean more opportunities to spoof your location.
We do not test each and every connection to ensure it is functional. This is one of the places where we have to assume companies are telling us the truth about their products. However, we do investigate if we find a server is unavailable during testing.
Most VPN companies offer servers in Asia (sometimes excluding China, as explained below), Australia, Canada, the US, and Western Europe. Better services include a few servers in Africa, Eastern Europe, the Middle East, South America, and South-East Asia. We give preference to services with robust offerings in Africa and South America, two areas generally underserved by VPN companies.
Unsurprisingly, there are limitations to this information as well. For the sake of clarity, we have not noted a difference between rented servers, servers owned outright by the VPN company, and so-called virtual locations. A virtual location functions like a server in a given country without having to physically be in that country. This won’t matter to most people, but if you’re concerned about having your data in a specific region, knowing which are legit servers and which are virtual can be an important differentiator.
Going forward, we plan to break down server information in finer detail.
We take particular note as to whether or not a VPN company offers servers in Cuba, China, Russia, and Turkey. These countries have particularly restrictive internet policies, although they are far from the only ones. Our understanding is that connecting to one of these servers from within the country will not circumvent censorship. However, it would provide some modicum of privacy and security to the user, particularly for visitors to the country.
VPNs are important tools in these countries and others around the world for individuals to access the web freely and without reprisal from the local government. Given those stakes, we believe it would be unethical to choose a service that would be “best” for circumventing censorship. The consquences for our readers if we were to get it wrong are far too high. We will not, for example, write a story about the Best VPN for China.
User Experience
There is a false dichotomy in digital security between a product’s ease of use and the value it provides. We frequently see commenters say that a particular product (generally one they have never used) must be worthless because it looks pretty. That ignores a very real truth about humans: no one is going to do a difficult or annoying thing just because it might protect them from hypothetical threats. A well-designed security product that average users can actually, well, use, is better than a perfect security tool that is only accessible via the command line.
When we review VPNs, we go through the setup process for each app. We also take time to poke around settings, and see how easy it is to perform certain functions. It’s important that readers, like you, have a sense of what using a given product will be like from reading our reviews.
User experience is important, but rating it can be tricky. Sometimes, an excellent user experience takes a mediocre product and makes it better. Sometimes, it undercuts the value of an otherwise stellar offering. In general, we place great emphasis on a product being easy to use, and accessible to all levels of experience.
At the same time, we cannot deny the importance of technical excellence, especially when it’s combined with value. Private Internet Access, for example, isn’t the easiest product to use. It is, however, one of the best VPN services on the market, thanks to excellent technology, commitment to privacy, and low monthly fees.
VPN Protocols
VPN is a mature technology, but it’s hardly a static one. There are several different means for creating a VPN connection (you can even create your own VPN), but not all of them are equal.
In our reviews, we give preference to the services that offer OpenVPN. This open-source protocol has been picked over by volunteers, helping to quickly find and fix potential issues. It also has a reputation among professionals for providing better speeds and more reliable connections. IKEv2 is another good choice.
The other protocols out there are either older or held in less high regard. With these availability of excellent tools, it’s a mark against a VPN company if they can’t offer them.
Some services, such as VyprVPN, have started to deploy their own VPN protocols. Most are built on establishing tools, so it’s not as foolhardy an idea as rolling your own encryption protocol (looking at you, Telegram). It is harder, however, to evaluate how well these proprietary protocols perform.
In the future, we expect the WireGuard VPN protocol to be increasingly important. For now, we consider it to be a pleasant bonus, but not a mainstream technology that’s ready for mass use.
VPN Speed Testing
Most of our readers seem concerned with the security of a VPN, but also the impact on internet speeds. That’s understandable, since most VPNs will increase your latency and slow your overall internet connection. Why this happens is simply a product of taking your internet traffic and running it through extra steps.
To find the fastest VPNs, we compare average speed test results from Ookla with the VPN active, and then when the VPN is inactive, in order to find a percent change. The Ookla test returns results for latency, upload speeds, and download speeds, so those are the metrics we use as well. In order to find an average speed for a given test, we run the Ookla tool five times, discard the top and bottom results, and average the remaining three test results.
In the first round of tests, we use an automatically selected VPN server and an automatically selected Ookla test server. This is meant to simulate what we believe average use will be, since most VPNs will automatically choose the nearest (and usually, fastest and lowest latency) server by default.
In the second round of tests, we try to simulate VPN use while connected to a distant server and under less-than-ideal network conditions. For these tests, we select a VPN server in Sydney, Australia (or as close as possible), and an Ookla test server in Anchorage, Alaska.
These tests have limitations. First, issues with the internet connection we use in testing could affect the results. For this reason, we perform speed tests in between results and compare it to historical speed data for our specific connection, to ensure that it is operating within expected parameters. Second, background system processes on our test computers could muck up test results, though we strive to avoid this.
Most significantly, despite gathering numerous test results, it is still only a single point of data and not enough to give a definitive judgment on a service’s overall network performance. Consider that when 10Beasts does our Fastest Mobile Network survey of wireless providers, we test constantly over the course of several days and across several states. To create a truly accurate picture of VPN performance, we would have to replicate that kind of testing as well which would be far more expensive, far more time-consuming, and requiring tools that currently do not exist.
Additionally, VPN performance may depend greatly on the VPN server you connect with. Some VPN apps have fine-grained server selection tools where you can choose a specific server, over and over again, but not all.
Because of these limitations, 10Beasts presents its speed testing not as the final word in a VPN’s performance but instead as a snapshot. It is meant to say that at this given day and time, this VPN performed this way.
Note, however, that we have been performing these test periodically for several years now, and that even these isolated points of data do help us build up a picture of each service’s performance over time. In the future, we hope to offer a comprehensive “look back” over several years of results for several VPN products.
Trust and Privacy
When its product is active, a VPN company has the same level of insight as your ISP has into your online activities. Because of that, it’s important that you trust the VPN company you sign up with, and that you are comfortable with its practices, as well as the potential pitfalls using a VPN might entail.
When we review VPNs, we make sure to read the privacy policy of each service. In particular, we are looking for logging policies—that is, what information the company retains about online activities. We also directly ask VPN companies what legal jurisdiction they operate under, how the companies make money, and what information the company gathers about user activity.
It is, of course, entirely possible for a VPN company to lie to the public in their privacy policy, and lie to us during our interviews. The goal with these questions is to put them on record, should contrary information surface in the future.
The steps that VPN companies take to protect your information vary. Some, like Private Internet Access, issue users a semi-random username, as part of an effort to obfuscate individual identity on their service. Others operate under legal jurisdictions that allow them to avoid retaining information, or handing it over to law enforcement. For example: ProtonVPN operates out of Switzerland, and NordVPN is under the legal jurisdiction of Panama.
Some consumers refuse to use VPNs based in the US, out of concern that these companies will be compelled to hand over information to law enforcement. It is worth noting, however, that the US does not have any mandatory data retention laws that would require companies to keep certain information on hand. The UK, however, does have such rules.
Some VPN companies operate out of Hong Kong and may be subject to government pressure to which we are not privy. Other companies may list their offices as existing in one country, but actually operate out of another. We think this information is important to report and include it in our reviews. We also acknowledge that the legal frameworks and ethical practices of other countries vary widely.
We are also wary of xenophobia in the guise of seeking the best and most secure option. Rumor mongering is not unheard of in the security industry, nor is using baseless fears over race, class, and other factors. For example: China and Russia have been accused of numerous cyberattacks against the US, and are known for fostering oppressive environments domestically. Because of this, some consumers refuse to use security products from these countries, believing that they are inherently compromised. But by the same token, the US government is responsible for the largest and perhaps most intrusive intelligence gathering operation in the world (if the information from Edward Snowden is to be believed), and has even intercepted domestically made products in transit to install malicious software. Yet US products are often regarded as more trustworthy—by US customers, at least.
For the time being, we are hesitant to penalize a product for its country of origin alone. Instead, we present the information we collect, provide context, and encourage readers to make their own choices. In the future, we hope to develop tools for our readers and reviewers to better assess the track record of individual companies and countries for security and privacy.
VPNs and Net Neutrality
The 2016 presidential election and changes to the FCC’s rules on Net Neutrality have spurred unprecedented consumer interest in VPNs. We’ve got the search traffic data to back that up. As a result, VPN companies have started popping up like mushrooms after a rain.
Many of these companies are respectable operations that take their responsibilities as stewards of user data very seriously. Others are fly-by-night companies that have existed for less than a week and might not be around much longer than that. Still others are outright scams that don’t even encrypt your traffic.
Our testing assumes that the VPN companies we review are good actors, operating in good faith. We rely heavily on the work of security researchers who have unmasked some of the worst behavior among VPN providers, and on the robust security community that is quick to point out the flaws in any product.
10Beasts freely admits that this is inadequate. What the VPN industry needs is similar to what is already common for antivirus products: robust third-party testing and easily deployed tools to confirm that the product is performing as advertised. We hope to encourage the creation and dissemination of such tools and standards in the future.
VPNs for Android and iOS
Smartphones have, for the most part, replaced laptops as the go-to mobile digital device. Instead of just connecting laptops to insecure Wi-Fi networks, you now connect your phones and tablets, too. So, as you might guess, we believe that evaluating VPNs on mobile platforms is important.
In general, we review mobile VPNs the same as desktop VPNs. Most of the features are the same, but we do take pains to highlight the differences. We also truncate our speed testing and skip the international testing which uses a VPN server in Australia. While our desktop VPN speed tests are carried out via wired Ethernet connection, the mobile speed tests are done over Wi-Fi. During these tests, we deactivate the cellular radio in order to reduce variables that could affect the results.
One major difference between mobile VPNs and desktop VPNs is the selection of available protocols. It’s much more common to find OpenVPN on Windows and Android, less likely in macOS VPNs, and outright rare on iOS. That’s because Apple requires developers to jump through additional hoops if they want to use OpenVPN in their iOS app, or in an app they want distributed through the macOS App Store.
Fortunately, more developers are taking the effort to include OpenVPN in their iPhone VPN apps. We try to reflect that extra effort in our reviews. However, this will hopefully no longer be a differentiator in the future and including OpenVPN will be the norm instead of the exception.
Another difference falls into the ease-of-use category. Mobile devices have a different design language, since you interact through a touch screen instead of a keyboard and mouse/trackpad. A really successful mobile product will be visually and functionally similar across all platforms, but tailored for each—whether it’s a VPN for Android or iPhone.
A Note About Ethics
In an era of fake news, phony reviews, and mounting concern over pay-for-play content, we believe it is important for readers to understand how our company earns money and how our reviews are written. At the top of every review on 10Beasts—VPN or otherwise—is the following statement:
- 10Beasts reviews products independently, but we may earn affiliate commissions from buying links on this page.
In practice, this means that 10Beasts the company may earn a commission from the company whose product has been reviewed or some other entity. This is entirely separate from our editorial process; reviewers do not have any knowledge of the specific ways in which reviews are monetized. Nor do reviewers receive a cut of that monetization. Reviewers, full-time or freelance, are paid for their work and do not earn a commission for the reviews they produce.
Importantly, companies, even those who have affiliate relationships with 10Beasts, do not dictate the outcome of reviews. Our reviewers value their reputations, and we would not stake them on what amounts to bribery.
The Evolution of VPN Testing
At 10Beasts, we strive for reviews that are meaningful, based on testing that is reproducible. There’s no point in testing that can’t deliver results that mean something. Similarly, there’s no point in including so much information in a review that a reader could not understand what’s important and what’s just so much techno-fluff. Walking this line is always a tradeoff, and for VPNs it is no different.
As always, we will adapt and improve our testing as well as our reviews as the products change, but also the landscape around them. Perhaps a new technology will completely upend what makes a VPN worthy. Whatever the case, the VPN reviews you read here on 10Beasts will always be as accurate and useful as we can make them.
