Whitelisting Vs Blacklisting

The fundamental principle that underpins all security is the need to stop bad people or processes while allowing good people or processes. So security is about access control; and  access control starts with identity. But identity on its own is not enough – we also need to understand purpose. We need to identify the person or process, and decide whether the intent is good or bad.

This article was written for, first published by, and reprinted here with the kind permission of
Infosecurity Magazine.

Consider passports in the physical world. They prove identity, but do not tell us intent: is the intent of that identity to do good or bad? We reinforce identity with lists of known intent: a whitelist of frequent flyers or VIPs whose intent is known to be good, and a blacklist of terrorists and bad people whose intent is known to be bad.

Cyber security is the same: based on identity and intent we maintain whitelists of known good (or at least acceptable) behavior, and blacklists of known bad (or unacceptable) behavior. Security is largely based on how we use these lists. In the main, we either allow what’s on the whitelist and prevent everything else; or we prevent what’s on the blacklist, and allow everything else. We tend to concentrate on one approach or the other: whitelisting or blacklisting.

Keeping our computers clean is a good example. In the beginning the anti-malware industry simply blacklisted the bad things. But now the alternative is gaining traction: whitelisting the good things. We need to know which is best for maximum security.

In favor of blacklisting

The basis of anti-virus security is a blacklist of all known malware. The technology is based on blacklisting because in the beginning there were very few viruses. A primary advantage of blacklisting is that it is conceptually simple to recognize a few bad things, stop them, and allow everything else.

A second argument in favor of blacklisting is ‘administrative ease’. The maintenance of blacklists is something we can delegate to trusted third parties – in this instance the anti-virus companies. They in turn, particularly with the advent of the internet, can automatically update the blacklist for us. Basically, we don’t have to do anything.

Dan Power, UK regional manager, Spamina

Whitelisting is different: it is difficult to delegate to a third party the decision on which applications we need. “Whitelisting would be the perfect solution if people only have one computer that is never patched and never changed,” explains Dan Power, UK regional manager for anti-spam company Spamina. “Intellectually it makes perfect sense to only allow execution of the files that you know to be good.” But maintaining this whitelist is difficult. “The problem comes when you have to register or re-register every DLL every time you install a new, or patch an existing, application. Which people do you allow to install their own software, and which people do you stop? And which bits of software can make changes and which can’t? It becomes more of an administrative rather than intellectual issue.”

David Harley, senior research fellow at ESET LLC, agrees: “Whitelisting – which isn’t much different in principle to the integrity checking of yesteryear, requires more work by internal support teams and interferes with the end-users’ God-given right to install anything they like; which is more of a problem in some environments than in others.”

David Harley, senior research fellow at ESET

That’s not to say that some people consider such delegation to be impossible. Last year Microsoft’s Scott Charney proposed a form of whitelisting for access to the internet; that is, only users with an internet health certificate for their computer should be allowed access. He has few supporters in the security industry. Power, again: “If computers were like televisions, with just one base operating system that was never changed, then it’s doable. But in the real world there are just so many variables associated with Windows and all the bits of software that have ever been written for Windows, that it’s almost impossible to be able to say what is and what is not a clean or healthy computer.”

Jennifer Gilburg, director of marketing at Intel

Jennifer Gilburg, director of marketing at Intel, sees a different problem with this type of whitelisting. “Think of e-commerce,” she said. “An online trader would rather take the occasional fraudulent transaction than risk turning away a good transaction. So the thought of blocking a user from coming onto the internet until they are trusted would terrify many of the e-commerce providers who make their livelihood on the basis of the more users the better. I suspect that most of the e-commerce world would be lobbying very hard to put down this version of whitelisting.” So one of the strongest arguments in favor of blacklisting is the problems concerned with whitelisting.

In favor of whitelisting

Henry Harrison, technical director, Detica

However, Henry Harrison, technical director at Detica, points to a specific problem with blacklisting. “Anti-virus blacklisting,” he says, “is based on the idea of detecting things that are known to be bad and stopping them. But it simply cannot detect things that are bad, but not known.” Zero-day threats are not known simply because they are zero-day threats – and blacklisting merely lets them in as if they were good. “What we are seeing today,” continued Harrison, “is a lot of targeted, covert attacks – external infiltration into corporate networks with a view to the theft of valuable information using techniques that are specifically designed to evade blacklisting – and one possible response to zero-day threats is whitelisting.”

Lumension’s senior vice president Alan Bentley, points to the sheer volume of malware as a problem for blacklisting. “Blacklisting,” he explains, “is threat centric. Whitelisting is completely the opposite: it’s trust centric. While blacklisting malware used to be adequate, the whole threat arena in the cyberworld has exploded to such an extent we now have to question whether blacklisting alone is still good enough.”

Alan Bentley, senior vice president, Lumension

This is what Lumension does: it protects end-points (such as the PC on your desk) by making it administratively easy to create and maintain a whitelist of acceptable applications while supporting that with a blacklist of malware. “We believe that if you look at the two things together, whitelisting should absolutely be the first line of defense for any organization, because it simply stops everything that isn’t approved. But what it cannot do is remove malware once it has embedded itself into a machine.”

Bit9, like Lumension, is a company that concentrates on whitelisting. “The premise of application whitelisting is very simple,” explains Harry Sverdlove, chief technology officer. “What you want running on your system is a much smaller set than what you don’t want. We apply this model to other aspects of security in our life. For example, who do you let into your home? You don’t keep a list of everyone bad in the world. Rather, you only allow people into your home whom you trust.”

Harry Sverdlove, CTO, Bit9

What we’re seeing is that the explosion in malware (in excess of 2 million new pieces of malware every month) is exactly what makes us question whether blacklisting remains realistic. “As a general rule, whitelisting is always more secure than blacklisting,” continues Sverdlove. “But it requires you to think more about how software arrives on your systems and whether or not it is trustworthy. That’s why a software reputation database can be an invaluable aid in whitelisting – it provides a trust rating on software, like a trusted advisor or background security check service, that can make the process more manageable. If everything you run comes from a well-known third party, approving software almost exclusively from a cloud based reputation service can be enough. In most cases, however, you also have your only custom or proprietary software. An effective and robust whitelisting solution allows you to combine both your own policies along with those from a reputation database.”

So we should ask ourselves whether we can harness the power of cloud-based reputation systems to generate our whitelists. Spamina already uses this methodology to produce its blacklist of spam sources, calling on six separate reputation blacklists, but never relying on just one (thus minimizing the chance of false positives).

The anti-virus industry

“I’ve never advocated AV as a single defensive layer,” says ESET’s Harley. “Whitelisting can and does work for businesses, though it works best where there’s an authoritarian IT culture, rather than laissez-faire: restricted privileges and so on. I wouldn’t generally recommend it as a complete substitute for AV, but if it’s implemented properly, it’s a rational multi-layering strategy. It does, at a stroke, obviate most of the risk from social-engineering-dependent threats. In fact, most AV nowadays does have some whitelisting ability, though how it’s done and to what extent varies enormously.”

Ram Herkanaidu, security researcher, Kaspersky Lab UK

Ram Herkanaidu, security researcher at Kaspersky Lab UK, has a similar viewpoint and acknowledges the increasing relevance of whitelisting. “As the amount of malware increases,” he said, “I can see at some point it could be more efficient to only allow whitelisted files to be run in an organization. The idea has been around for a while but many things have to be taken in consideration, like software updates (especially windows updates), remote users, smartphone and non-standard users.  Ideally as well as using the vendor’s whitelist you could have a local whitelist too.  So while the idea of having a, ‘trusted environment’ is very appealing, in practice it is difficult to achieve.”

Kaspersky, like other AV companies, is already looking into whitelisting. “We have been running a whitelist program to collect information about all known good files,” continued Herkanaidu. “The files are sent to us by our whitelist partners and also through our Kaspersky Security Network (KSN). This is our ‘neighborhood watch’ which users become part of when they install Kaspersky Internet Security. Information about all unknown files is sent to our ‘in the cloud’ service and automatically analyzed. If malicious, all computers within the network are protected. If it is not malicious it will be added to our whitelist.  This has two benefits for our customers: it will reduce the risk of false positives, and will increase scan speeds. In this way we have been able to collect information – not the files themselves – about millions of files.”

Whitelisting or blacklisting?

So what’s our conclusion? Whitelisting is fundamentally the better security solution. If something isn’t on the list, it gets stopped – the default position for whitelisting is secure.  But with blacklisting, if something isn’t on the list it gets allowed – the default position for blacklisting is insecure. Against this, the administrative effort involved in blacklisting is minimal compared to whitelisting; and the difference increases as the size of the whitelist increases. However, the efficiency of blacklisting decreases as its size increases. You could almost say that whitelisting is best with a small whitelist, while blacklisting is best with a small blacklist. However, since neither of these situations is likely to occur in the real world, our conclusion is simple: you need both.