Old Mac Bloggit isn’t really a grumpy old man…
…he’s really a rather nice young chap. But he’s certainly feeling a bit peeved right now, and with some reason. He’s upset about the unquestioning articles in the New York Times (31 December) and the Register (1 Jan) discussing a new report by Imperva. Actually, I discussed it in Infosecurity Magazine on 28 November.
Imperva concluded that anti-virus products are not that good (“The antivirus industry has a dirty little secret: its products are often not very good at stopping viruses,” says the NYT). Imperva’s proof is that VirusTotal (an online collection of AV engines) failed to block many of the 0-day viruses it threw at it. What I said in Infosecurity was that “the real value of VirusTotal is in allowing users to check whether a suspect file is actually malware – it was designed to check malware, not to check AV products.”
Mac Bloggit doesn’t have to acknowledge the niceties of journalism, and can be more succinct. “Perhaps the NYT would care to look up the terms heuristic analysis, behaviour blocking, sandboxing, behaviour analysis, whitelisting, integrity checking, traffic analysis, and emulation, among other approaches that a security program might use to detect possible malicious activity.” His point, and he has a point, is that VirusTotal does not and cannot measure the efficiency of these parts of AV products. The fact that Stoppem Anti Virus on VirusTotal doesn’t detect the latest virus doesn’t mean that Stoppem Anti Virus on a PC won’t detect and/or block the very same latest virus.
Using VirusTotal to judge an anti-virus product isn’t merely bad form, it is positively dangerous – it might tempt users into abandoning AV altogether. That would be a very, very bad idea. The Imperva report is actually a sleight of hand by a non AV vendor. But here’s the rub: the AV industry isn’t innocent of its own sleights of hand.
The one that gets me personally rather hot under the collar is the ‘destroys all known bacteria dead’. Well, that’s the clear message. The actual terminology is ‘stops 100% of viruses in the Wild’. What it is really saying is that Stoppem Anti Virus detects every virus in the Wild List. And the Wild List is very different to ‘in the wild’. In fact, the Wild List is effectively compiled by the AV industry; so in reality, any AV company that doesn’t score at least 99.99% success against viruses in the Wild is largely incompetent.
So I would say this. Imperva, you have been a bit naughty in your report. AV industry, you can be a bit naughty yourself. So stoppit, both of you. Anti-virus is good, not perfect, but essential. Just tell us the truth.
David Harley includes quite a lengthy comment on this blog in his post, Going beyond Imperva and VirusTotal. In particular he delves into the pros and cons of WildList testing. He doesn’t completely disagree with me; but nor does he completely agree – so it’s well worth a read.