How we test VPNs

For us to be able to give good advice to people looking to find the best VPN for them, first of all we have to review the products. With so much to consider beyond what the marketing pages on provider websites offer, this is a big task—and an important one, because people are entrusting their online privacy to these VPNs.

What's more, many people have different priorities when it comes to choosing a VPN —some might want first-class streaming performance, while others might only want to pay for a service in cash. Beginners might be intimidated by techy designs and want a simple interface, while seasoned users will appreciate a more complex setup.

Here, I'm going to outline every step that our team VPN experts and I take when reviewing products, and also how I decide which providers, like NordVPN and ExpressVPN, to recommend to people on the search.

Our VPN testing ethos

The most important part of our testing is the fact that we never take anything at face value. Is a VPN provider claiming it can unblock dozens of Netflix regions? We'll test that ourselves. Claiming a feature has been independently audited successfully? We're checking that report for ourselves.

Essentially, we use every feature first-hand to make up our own minds on if it's good quality, and try to break as much as we can to make sure it stands up to not only daily use, but extraordinary circumstances as well. Very few of our readers will care about every aspect we check and test, but we structure our testing procedure and subsequent reviews in a way that is intended to help both new and advanced users get the info they need in order to form their own opinion.

Of course, opinions on software are just that: opinions. Some of the sections in our reviews will inevitably be subjective, and our own taste certainly comes into play when we evaluate things like ease of use and app design. However, where we can we lay out the facts in an unbiased way so that our readers get a 360-degree idea of what a VPN offers and is like to use. After that, the decision is yours.

The initial factfinding process

We start off by scouring the VPN provider's website, where we note down headline figures like how many servers it providers globally, and where they are.

While network size doesn't matter as much as VPN providers claim it does (there are VPNs with fewer than 2,000 servers that can compete with others that have five times that), it's an important metric to find out initially. 

Just as important—if not more so—is the number of server locations available. Having servers in dozens or even over a hundred countries generally shows decent investment in the network, and also  means more people worldwide can use the service with ease. For many users, though, it generally boils down to whether there's a server in their home country.

General server information on things like which protocols are supported (OpenVPN and WireGuard or a similar modern protocol are a must), whether P2P is supported and on which servers, and what streaming sites are claimed to be unblocked are also good to note at the start of the process.

Knowing these claims allows us to set our standards suitably high—if a VPN claims to support torrenting but in reality that's only on a handful of servers, that's a disappointment. The same goes for streaming unblocking—if claims to unblock Netflix Japan but fails in all of our tests, that's arguably worse than never promising anything to begin with.

Good platform support is at the core of any good VPN, as in the mobile-first world, more and more people are relying solely on handheld devices to live their online lives. 

Just about all the top providers you'll see featured on Tom's Guide will have a Windows VPN, Mac VPN, Android VPN and iPhone VPN app available. However, while lots of providers claim to support a huge range of other devices, once you head to that page on the site you'll simply get a walkthrough of a manual setup, rather than an app download.

We also dig a little deeper than just seeing what protocols are available—we also investigate how each one is implemented, too, checking out authentications and encryption methods. This sort of stuff is usually available on-site, but finding it can often be a challenge.

Beyond basic privacy, plenty of VPNs offer extra features, which are no doubt plastered all over the site. These often include blockers for ads and malware, Multi Hop servers, tunneling through Tor, split tunneling, plenty of others. This info is great for guiding our testing when we eventually get stuck in.

Everyone loves a cheap VPN, but the prices on offer from some VPN providers can be a little deceiving. We make sure to note down exactly what you'll pay for every single plan—longer plans are the best value, but usually require a large upfront payment—and almost all VPNs crank up the price on your first renewal.

This (fairly arduous) data collection process sets us up with a solid base to guide our hands-on testing process, and also is a good litmus test for a provider that cares about what it's doing. 

If there are tons of outlandish claims, a general lack of detail, or it's hard to find things that really should be front and center like privacy policies, that gives an overall impression that said service might not be up to scratch in other areas. In general, the most reliable providers will have well thought out websites that are transparent and honest with their claims and promises.

Testing the privacy fundamentals

Privacy is undeniably the most important aspect of a VPN—if it can't keep you safe online, all the bells and whistles are useless. However, actually assessing this can be a tricky task since much of the information has to be taken on face value from the VPN itself. There's still plenty we can do, though.

The encryption protocol comes first. WireGuard and OpenVPN are the very best, IKEv2 is a decent alternative, while L2TP/IPsec is only useful for niche cases and PPTP is old, insecure, and should be avoided.

AES-256 data encryption is the norm and, along with ChaCha20, should be the minimum offered. RSA-2048 or 4096 should generally be used for handshaking, while Perfect Forward Secrecy—essentially this means the key is changed every session—is a big tick. While these are generally followed by the big providers, many VPNs don't fully explain how their encryption works, while others are transparent, and even offer extras like NordVPN's Onion over VPN. 

If it's tricky to find this out, we typically get in touch over email or with a live chat operator, which often gets us some answers. We'll also download sample OpenVPN config files to investigate exactly how it works behind the scenes.

The app features are really important, too. Ideally, we'll look for first-party DNS servers, and robust IPv4 and IPv6 DNS leak protection—this is essential for making sure your online activity isn't visible to anyone else. A kill switch is also just about a non-negotiable so that you can stay safe even if your VPN cuts out. 

However, something important to consider is the fact that not all platforms and devices have the same functionality—mobile apps especially. To make sure we're not making any inaccurate sweeping statements like "this VPN has a kill switch", we test on each platform and outline any areas where they may fall behind.

What's more, our kill switch stress-testing is one of our most savage processes. We use tons of low-level tricks to try and find just about any way of getting the kill switch to fail—and, surprisingly often, they do. The ones that make it through this process are the best of the best, and while they're perhaps not rigors the application will face day-to-day, it's a true test of the code's stability.

If you want a really simple test, all you need to do is connect to your VPN, and switch off your router. Does it realise what's happened? Have you been warned of this? Does it reconnect automatically when you're up and running again? Quality providers will do all of these things—a bad one might not even realize you're offline.

Fact-checking any logging or no-logging claims

When you're connected to a VPN, all of your internet traffic goes through its servers. Being able to trust that your VPN isn't collecting that data and selling it is a key part of a quality VPN, so seeking out a 'zero-logging' VPN is a must. The VPN companies know this, and plaster claims of zero logging all over their websites—but things aren't always as cut-and-dry as it seems.

Privacy policies are key to making your decision, and a clear and well-written one is possibly the biggest green flag you can get. If it's incredibly long and barely comprehensible, or just a few sentences that tell you very little, we begin to get a little worried.

What we look for is a document that's easy to read, and that helps you understand what the privacy policy actually means. However, this is still all based on trust—although independent audits go a long way in bolstering that.

Checking the small print is an essential part of our review process as multiple times we've found providers admitting to some logging taking place. and, besides what the policy says, it's a good benchmark for how honest and open a company is.

Analyzing any security audits

I mentioned audits above, and they've become a big thing in the VPN industry. Essentially, a specialist firm—PricewaterhouseCoopers, Cure53, Deloitte, KPMG, and F-Secure are some of the big gest names—is invited to test an aspect of a VPN. This could be the privacy or zero-logging policy, an app release, an extra feature like ExpressVPN Keys or NordVPN's Meshnet, or the product more generally as a whole. 

As you can imagine, this is a considerably invasive process, and opens the VPN up to some significant bad press if major flaws are found. therefore, I commend any VPN brave enough to go through it—although it's now becoming the norm.

However, every audit is different, and we take a close look at the scope of each and every audit that's released. Some are very valuable, others, not so much.

For example, sometimes you'll see an audit solely focused on, say, the browser extension, which isn't much to shout about. Others, like TunnelBear's infrastructure and website audit or NordVPN's privacy policy audit are much more impressive—especially when they return excellent results.

We also check the depth of the audit. Some providers only provide access to the apps with no extra info—known as a 'black-box audit'—while others allow full access to the network, source code, and just about everything else—known as a 'white-box audit'. Clearly, a white-box audit is better, as this allows testers to scrutinize every last line of code.

Finally, the age of an audit is very important. Within the last year is always a good yardstick, and many providers re-audit every year. An audit from three years ago will likely be out of date, and certainly won't have considered features added since then. for us, the more the merrier.

Putting performance to the test

To many, having a fast VPN is incredibly important. After all, no one wants a VPN to slow down the superfast connection you've paid good money for, so we spend lots of time seeing how they perform in the real world.

We test in two locations, both of which have access to a 1 Gbps wired Ethernet connection. The first is a home in the US, and the second is a Windows 365 cloud PC hosted in the UK.

To start, we test the connection with the VPN disconnected to get a baseline, using at least three different speed testing tools to measure it. These include nPerf, the SpeedTest website and its command line app, SpeedOf.me, and Netflix's Fast.com. Each test is run five times and we use the median figure.

Then we connect to the nearest server using the best protocol available, and repeat the process. If a provider supports more than one protocol (WireGuard and OpenVPN, for example), we'll check both using the same process. 

So far, the process has gone through at least 45 tests—likely far more—but we don't stop here. We do this in the morning and evening to get an accurate representation, in case network issues are present in either session.

It's good to keep in mind that the speed testing is probably the least concrete part of our testing process—they're a snapshot in time rather than a live representation—and, of course, we can only test from the locations we're based in. If you live in Asia and you're connecting 3 months after our test was run, results may vary.

However, despite its flaws, testing the speed of a VPN in this way does give a good representation of what a VPN is capable of, and also gives us insight into how reliable the servers are. If we're seeing wild fluctuations in speed every time we test, that's not a good sign—and we'll tell you about it.

Unblocking streaming sites

How a particular provider is as a streaming VPN is key to a lot of users—many aren't interested in privacy at all and simply want to watch the favorite Netflix shows when they're abroad. Testing this accurately, then, is absolutely key.

Lots of popular VPNs promise to be able to unblock any number of services and locations on their websites, while others choose to focus their branding on privacy and shy away from anything streaming-focused. 

However, what the website says often doesn't match our testing. We've found that providers like NordVPN who rarely say anything about streaming are world-class, and the noisiest brands often fall short. In general, it's better to underpromise and overdeliver—and seeing exactly where a VPN claims to unblock is a good start.

If a VPN does make promises, you've definitely got a better leg to stand on in the case it doesn't work. If you ever find yourself in that situation, the support team should be able to give you some advice on getting back up and running again.

To test these claims out, we test to see if Netflix is available from a number of servers in different countries (UK, US, Canada, Australia, India, Japan to name some), and note down the results. 

We also test Amazon Prime Video and Disney+ from US VPN servers, BBC iPlayer from UK VPN servers, and a few other regional streaming services like Austria's Servus TV and Australia's 9Now and 10Play.

If there are any inconsistencies with our testing and any claims made on site, we'll always follow up with the support team—and although this might seem like a bad sign, the VPN provider can still redeem itself.

The way this is all handled is key. If we're directed to a particular server that works, that's great, and even if they can't do this, we always appreciate the offer to log the issue and follow up on it. If it's essentially "sorry, good luck", that's a red flag and will be noted as such in our review.

Apps design and functionality

A VPN's range of apps is absolutely critical. No matter how well-maintained the servers are, or how rock-solid the privacy policy is, if it's a nightmare to actually use that's pretty much all for nought. 

The very minimum we'd accept from a VPN provider is to have apps for Windows, Mac, Android, and iOS—the vast majority of users will primarily be using one of these operating systems, so coverage is key. Support for others systems like a Fire Stick VPN or Linux VPN isn't so important, but the more the merrier.

If a VPN has apps for plenty of different systems, it shows us it's taken the time to cater more more users, and has the resources necessary to actually run a quality service.

Browser extensions are also nice to have, and while even some of the biggest VPNs don't offer these, they're still nifty tools. Most are merely proxies, although ExpressVPN's Chrome VPN extension actually controls the apps that's installed on the device.

App design and usability can be very subjective things, and while we may have our preferences, they could be different to yours—we'll present the facts and offer our opinion, but you're in no way obliged to agree.

However, if a VPN has a tricky interface that hides useful features away, or makes it hard to do the simplest things like actually find and connect to a server, this will absolutely affect its score. Alphabetizing the list of servers is an incredibly easy way of making this process as simple as possible, but you'd be surprised at how many well-known VPNs don't even do this.

Things like a favorites system and information of server load are excellent usability features, and garner healthy green ticks.

Useful but unobtrusive notifications are also a must, as well. After all, if you click connect and then head to your browser, how are you going to know if you're actually protected? On the other hand, intrusive notifications, especially when you're not using the app, are near unacceptable.

Many VPNs offer an automatic mode which selects what it thinks is the best or fastest server available to you. However, these aren't always perfect, and may actually deliver suboptimal selections, so we always check to see how well these features work.

Connections speeds aren't the only area we measure, either. We test how long it takes for a VPN to connect, benchmark DNS queries, the speed which websites actually load in the real world, any hiccups in streaming, and generally the wider performance of a VPN in use.

There are also some key features and setting we really want to see. For example, a kill switch is essential, which protects you even in the event that your VPN's connection drops by cutting your internet. Auto-connect features are useful as well, which allow the VPN to connect when you end up on particular networks.

Anything else on top of that—custom scripts, malware protection etc.—are nice to have and we'll note them in our review, but they won't have that much of an impact. Really, we're interested if a VPN is pleasant and functional to use for real-world users.

Customer support and written help articles

No matter how well-built a VPN is, they're complicated software and can all run into tricky issues that are hard to diagnose and fix on your own. That's why we place a fair amount of weight on how good a provider's support system is.

Typically we'll split this into two: written articles (or a 'knowledgebase') and support from a real person.

First up are the setup pages. We expect a quality guide for every operating system the VPN supports, and ideally we'll see additional tips for different versions and releases. The more the better here, and it's also important that the guides are easy follow.

The knowledgebase is the first port of call for just about any issue, so having a clear structure, a good search function, and well-written pages that make problems and solution clear and easy to fix is important. 

When testing this, we'll search for fairly broad terms (like 'speed', for example), and see what we're shown. Few hits in the results is a poor performance, and if we head to said pages and the advice is slim and unhelpful ('turn it off and on again'), this shows the provider hasn't put much effort into this. 

Conversely, if we get lots of hits, and the advice is more comprehensive that we expect, it's a really good sign that the VPN cares about its users, and is supporting them to get the most out of the software.

However, written articles can't solve every issue, so having some form of live chat support—ideally 24/7—is a must. We mark down providers for not offering this as it's a key form of support for every user, but email support is also very valuable too. It allows you to explain your issue more fully, and we've found that we generally get more comprehensive answers that solve issues first-time this way.

We test out both of these with at least one question with every provider we review to see how easy it is to get through to a real human—almost all live chat platforms start off with a bot, but good ones refer you to a person very quickly. 

Just like our speed testing, this is a snapshot of a service, but it does give a good idea of a company's ethos towards its users. If the investment has been made to quickly and effectively solve user issues, that's better than curt and cursory responses, all of which gives us a better representation of the VPN provider as a whole.