Anti-virus is essential – it’s just not as good as they tell us
Cyveillance is a company that knows what’s happening on the internet. That’s what it does; intelligence is its business. “Cyveillance,” says Eric Olson, Vice President, Solutions Assurance, “is built on a foundation of home-grown technology for monitoring and sifting through the open source or publicly available internet – in short, we hoover up anything we can from the public internet; and that produces a very large, noisy, unstructured and mostly pornographic haystack out of which we have built a system that not only collects the haystack, but also finds the needle. That needle can be just about whatever we or our clients care to define as a relevant needle. So our capability, if you like. is to do what you could do sitting in any cybercafé just poking around on the internet – it’s just we do it on a grander scale.”
And how grand is quite mind-blowing: information is collected from more than 200 million unique domain names, 190 million unique websites, 80 million blogs, 90,000 message boards, thousands of IRC/chat channels, billions of spam emails, tiny URLs and more. The intent is to provide the intelligence that allows proactive security – the ability to recognize a threat before it becomes a problem, and to solve the problem before it becomes a disaster.
Eric gives two examples. A bank might hear about credit card numbers being found or traded on the internet. It might go to Cyveillance who would then look for credit cards in its haystack, and would be able to tell the bank, ‘hey, these cards of yours have been compromised – you should cancel them.’ That, says Olson, “is a data-level example of what we do. An intelligence-level example might be the tweet or blog posting that says ‘I just lost my job, I told my wife and she left me, my kids hate me, I have nothing left to live for, I’m going back there with my Kalashnikov to kill everyone I used to work with’ – you would be surprised how many of those we find in a day. So if that company or executive is a protected customer of ours we will call their Head of Security and say ‘hey, somebody is threatening to come back to the office with a machine gun, and you might want to be on the lookout for that’.”
In short, Cyveillance is all about proactive security. “We provide realtime intelligence,” says Olson, “about cyber threats, malware, exploits, viruses, social engineering, scams, spear phishing etcetera ad nauseam early enough for a response to be proactive rather than reactive. We move the defensive perimeter out into the wild and allow an intelligence-led approach to security — more predictive and proactive than reactive.
But there is a problem. The whole security industry has become mired in the concept of reactive security: it’s what most of us believe security is all about. But reactive is never good enough: closing the stable doors after the horse has bolted has never yet stopped the horse from bolting.
So periodically Cyveillance does something to demonstrate the weakness of the reactive approach: it tests the leading anti-virus products against a current selection of malware samples it has found in its haystack. And the AV products are usually found wanting. The methodology is very simple: the test solely uses malware samples discovered by Cyveillance ‘in the wild’, and only those samples that are confirmed as malware by at least three of the tested AV products. For the latest test, this process provided a data set of approximately 1700 malware samples collected over a 3 day period in April.
Cyveillance wanted to make things reflect real life as far as possible; so it installed the AV products on PCs and then tested them against the viruses. Nothing special – just how things happen in real life. But those of us accustomed to seeing AV test results in the high 90% success bracket – if not VirusBulletin’s VB100 certification – should prepare for a shock. ESET’s Nod32 product was by far the most successful – but still detected just 38% of the viruses. The average detection rate for the 15 products tested was less than 19%.
I asked Olson why the anti-virus products were so poor at detecting viruses. “It’s really quite simple,” he replied. “If I’m a criminal and I make my money out of writing malicious code, I’m not going to waste my time releasing my malware into the wild if I’m not going to get a return on my effort – so I spend the $30 dollars or whatever to buy a copy of each these AV products and I test my code against the AV products. If they stop my malware when I’m getting ready to release it, then I rewrite the code until they don’t. So the answer to why these detection rates are so bad is not because the people at ESET or Sophos or Symantec or McAfee are stupid – they’re not – they’re clever, hard-working dedicated people – the problem is that all the protection they offer in desktop or network AV programs is also available to the criminal. That criminal will buy a copy of Norton, run it against his stuff, and if Norton fields it, he fiddles with his code until it doesn’t.” In reality, the criminal doesn’t even need to pay out for the anti-virus programs. “There are also various foreign equivalents to VirusTotal,” continued Olson. “If you upload a binary to VirusTotal that isn’t detected by the AV products, VirusTotal shares the sample with the industry – but there are dozens of equivalent foreign sites that don’t do this.”
This ability to write new malware, or adapt existing malware, to beat the AV products is well-known. So Cyveillance felt that the real test for the anti-virus products was to see how quickly they could recognise and catch the failures; so it retested all the AV products regularly over a 30 day period to see how quickly they caught up with the malware. The results can be seen in the following graphic (click for a full-size version): but the bottom line is that the AV products are still not that good.
Cyveillance tested thirteen popular AV solutions to determine their detection rate over a 30 day period and found that popular solutions only detect an average of 18.9% of new malware attacks. By day eight, AV solutions average a 45.7% detection rate. This rises to 56.6% on day 15, 60.3% by day 22, and 61.7% after 30 days. Top AV solutions take an average of 11.6 days to catch up to new malware. Since this does not include malware signatures undetected even after 30 days, users should not rely on the AV industry as their only line of defense.
I asked Olson if he got much negative feedback from the AV vendors. He said they had some interest when they first started doing the tests, but that they simply explained what they were doing, and how they were doing it, and nobody could complain. “As a company, our principle interest is not viruses – we’re not binary crackers – our principle interest is in studying what is happening out on the internet right now; and one of the things that happens is that we download all of this malware. The key point about our methodology and studies about malware being distributed in the wild is that we are doing what users do, not what virus crackers in control of their own standards and environments do – we’re clicking on links and visiting pages and getting infected like normal users – the only difference between what I’m doing and what my mom does is that I’m doing it on purpose; I’m trying to make it happen and she’s trying to avoid it; and my mom relies on one or more of those products we’ve just pilloried to keep her safe from all of those things, and they’re doing a not very good job of it.”
So, I asked Cyveillance, Why did you do this thing? “We’re not suggesting that you don’t use the tools that exist out there,” said Brian Hedquist, Cyveillance Director, Marketing and Communications. “But from an enterprise point of view we’re suggesting that you have to take other methods of protecting yourself – the AV companies have bandied their solutions around for so long that enterprises have got complacent with their firewalls and anti-virus – but the bad guys are getting better and better all the time and companies can’t just sit around thinking they’re protected when they’re not.”
“Anti-virus is absolutely necessary,” agrees Olson. “It is necessary but not sufficient. Companies have to get away from the reactive security mindset. Proactive security is possible when you have the intelligence, and Cyveillance has that intelligence. The technology arms race always favours the bad guy; so you have to train your employees; and train your executives who may be targeted on a personal basis one on one; and thoroughly research social engineering attacks. So, why did we do this? Partly because we believe that AV is necessary but insufficient; and partly because some of those other things that can help with proactive security, we can offer.”