Archive
Is the AV industry showing signs of exasperation at users’ apathy?
I wrote about the March 8 deadline for remaining DNSChanger victims to get clean or lose their internet in Infosecurity Magazine: DNSChanger poses a new threat to its victims.
But I had two late comments from anti-virus people currently in the States and separated by the trans-Atlantic time difference. They both echo Graham Cluley of Sophos’ comment that “if this is the only way to wake the affected users into sorting out the problem, so be it.”
Panda Labs’ Luis Corrons used remarkably similar language. “At least this will make affected people react and secure their computers,” he told me.
And ESET’s David Harley said, “Pragmatically, I don’t have a problem with this: law enforcement doesn’t have a specific responsibility for maintaining service for infected machines.”
But reading between the lines, I suspect that any anger is really directed not at the infected users being apathetic with their own security, but that the nature of the infection makes further infection likely. Such users are being apathetic with other users’ security; and that’s really not on.
The UK Cyber Security Strategy – Protecting and promoting the UK in a digital world
“The UK will become the best country in the world for e-commerce, the prime minister has promised.” His promise includes “a raft of measures to boost internet use in the UK, including a £1bn drive to get all government services online [within three years] and £15m to help businesses make the most of the web.”
This is not from the new UK cyber security strategy published by the Cabinet Office last week. It came from Tony Blair in 2002. And it didn’t happen.
I always worry when government says we will abide by the rule of law. Since government makes the law, it is the same as saying ‘you must do what we tell you’. Last week, the Cabinet Office explained that “Our vision is for the UK in 2015 to derive huge economic and social value from a vibrant, resilient and secure cyberspace, where our actions, guided by our core values of liberty, fairness, transparency and the rule of law, enhance prosperity, national security and a strong society.”
Cameron is perhaps less ambitious than Blair, allows more time (four rather than three years) and is focused on security. But is the end any more achievable?
I doubt it – and offer a few observations. Firstly, by far the majority of security companies and their experts have openly welcomed and praised this report. They have no option. The power of government purchasing makes it difficult for any business to openly criticise government. Indeed, the report acknowledges this lever:
To ensure smaller companies can play their part as drivers of new ideas and innovation we will bring forward proposals as part of the Growth Review to help small and medium sized enterprises fully access the value of public procurement.
However, regardless of what they say in public, many of these security experts have serious doubts. One, whose company statement had him praising the initiative, privately mailed me worrying about how many different government departments, quangos, committees, off-shoots and different law enforcement and intelligence agencies are involved in this strategy. It is always the joint that provide weaknesses, and this strategy has many joints.
Amichai Shulman, CTO, Imperva
The second observation, which to his credit, has also been highlighted by Amichai Shulman, CTO and co-founder of Imperva, is that there is no emphasis on protecting the individual.
The strategy has given only a few insights on how government is going to help businesses and individuals protect themselves. In fact, it has taken the traditional approach of non-intrusive, general advisor for tasks left to the individuals to do, e.g., keep safe and stay current with the latest threats. As we know, most consumers and enterprises don’t do that which explains why we’re in the cyber crime mess we live in today.
Amichai Shulman, Imperva
It would appear from the report that the government expects its GetSafeOnline website to be sufficient to protect the public. (You can see my attitude to GetSafeOnline here: UK Internet Security: State of the Nation – The Get Safe Online Report, November 2011.) I have serious doubts about its effectiveness. But I am more concerned there is no mention anywhere in the new cyber security strategy report of an existing CPNI-inaugurated initiative that has the potential to help the individual: the Warning Advice and Reporting Point, or WARP.
The WARP project is stagnating if not contracting. But the concept is still good. Given the right input and impetus WARPs could develop into a form of security-based social networking system, where individuals would share threat experiences between themselves, learn about new threats, and automatically report them back up the line eventually to CPNI. By sharing their information, by warning others, by offering help and advice to colleagues within any particular WARP, the individual security stance becomes much stronger.
This approach could help protect home computers from being recruited into botnets; and fewer active botnets means a more secure national infrastructure. I am worried that if the new strategy isn’t aimed at protecting the NI by protecting the individuals, how else is it to do it? Possibly by ramping up co-operation with and control over the ISPs. We will, says the report
Seek agreement with Internet Service Providers (ISPs) on the support they might offer to internet users to help them identify, address, and protect themselves from malicious activity on their systems.
It is too easy to move from this position to one of getting the ISPs to cut off infected users until they can prove their system is clean.
But it’s not all depressing; I have always known that there are comedians in government. Always leave them laughing when you say goodbye. And this report does just that:
- the Ministry of Justice will develop ‘cyber-tags’ as a form of online ASBO
- police forces are to recruit ‘cyber-specials’ (the internet traffic warden?)
- ‘kitemarks’ to help consumers distinguish between genuinely helpful products and advice and the purveyors of ‘scareware’.
- “…partnerships between the public and private sectors to share information on threats, manage cyber incidents, develop trend analysis and build cyber security capability and capacity.”
CESG share intelligence with the private sector? Now that one really made me laugh.
Imperva
The UK Cyber Security Strategy – Protecting and promoting the UK in a digital world
Shadowserver’s new anti-virus test suite – how good is it?
“Our little gnomes in the backroom,” says the excellent Shadowserver in an announcement headed ‘New AV Test Suite’, “have been working feverishly for the last several months to put the finishing touches on our new Anti-Virus backend test systems.”
Malware testing, as we know, is a tricky business. AMTSO, the Anti-malware Testing Standards Organization, has expended much energy and expertise in developing detailed methodologies designed to ensure fair, unbiased and accurate anti-virus tests. But do we get this from Shadowserver? Do we get a new AV comparison source that we can realistically access for accurate unbiased information on the different AV products available to us? Let’s see.
Shadowserver starts off with a fair comment.
No single vendor detects 100%, nor can they ever. To expect complete protection will always be science-fiction.
That being said, it goes on…
…you can see the different statistics of the different vendors in our charts.
Here’s a couple of examples.
Shadowserver's results for Avira
Shadowserver's results for Panda
The one thing that really leaps out here is that Panda apparently misses (shown in green) far more of the test samples than Avira. This is counterintuitive. Panda is a commercial product backed by one of the world’s leading security companies. Avira, which I personally trust sufficiently to use on my XP netbook, is a free product. Shadowserver provides a partial answer:
The longest running issue has been our inability to use Windows based AV applications. We can now handle that, however it is still not what you might buy for home or commercial use. We are utilizing a special command-line-interface version from each of the vendors that we are using. This is not something you can purchase or utilize normally. These are all special version but most of them do use the same engines and signatures that the commercial products use.
Luis Corrons, technical director, PandaLabs
This is important. Luis Corrons, technical director at PandaLabs elaborated:
What ShadowServer does is not an antivirus test. As they say, they do not even use commercial products, but special versions. Furthermore, it is static analysis of files they capture. It is a statistic. But the data cannot be used to say “product x detects more than product y” or “product x detects this percentage” as they are not using any of the other security layers used in real products (behavioural analysis/blocking, firewall, URL filtering, etc). The most you can say with this system is product x was able to detect y percent of files using their signatures and heuristics (the oldest antivirus technologies).
This is important. The AV companies have long recognised that the original signature database solution to malware cannot match the speed with which new signatures are required for polymorphic virus families. So they have supplemented their signature detection with more advanced and sophisticated methodologies.
In our case (Panda) ShadowServer is using an engine which is a few years old (at least 5) and of course is not using the cloud, so I can guarantee that our results are going to be awful. We have been asking SS for years to use a new version, but they were not supporting Windows. Now that they are supporting it, they forgot to mention it, but it’s not a problem as we’ll be sending them a new version with cloud connection. Anyway, even though in that way the results will be way better, or even if we are the number 1 vendor, that doesn’t mean anything, as it is only a static analysis of some files.
One solution would be for Shadowserver to work more closely with AMTSO. Shadowserver is not currently a member of AMTSO. I urge it to join. And I urge AMTSO to waive all membership fees so that this non-profit free service organization can do so. Both parties would benefit enormously. In the meantime, I asked David Harley, a director of AMTSO and research fellow at ESET, for his personal thoughts.
David Harley, senior research fellow at ESET; director at AMTSO
Shadowserver has never been discussed within AMTSO, that I remember… In the past they’ve shied away from suggesting that their statistics are suitable for direct comparison of vendor performance. One of the reasons they cited for that is that their testing has been focused on Linux/gateway versions, and you can’t assume that desktop versions will perform in the same way across a range of products. Including some Windows products will make a difference in that respect, but I can’t say how much, because I don’t know which versions they’re using. Where gateway products are used, it’s unlikely that the whole range of detection techniques are used that an end-point product uses. Detection is often dependent on execution context, certainly where detection depends on some form of dynamic analysis. A gateway product on an OS where the binary can’t execute may not detect what its desktop equivalent does, because the context is inappropriate. On the other hand, the gateway product’s heuristics may be more paranoid. Either way, there’s a possibility for statistical bias…
This isn’t a criticism of Shadowserver, which does some really useful work. I just don’t think I could recommend this as a realistic guide to comparative performance assessment…
Neither Luis nor David are known to shy away from the truth, whether of themselves or their products. But both seem fairly clear: Shadowserver is good; but this service is not yet ready. Shadowserver’s AV test suite will not give a realistic view of different AV products’ actual capabilities. Not yet. It needs more work. I’m certain that will happen. But for the time being at least, don’t use Shadowserver’s statistics to form an opinion on the relative merits of different AV products.
————————
UPDATE from Shadowserver
It is difficult to not compare one vendor to the next due to how we have the data
structured on the pages. It would be impossible not to try and derive conclusions
from those results. While that is the case, our goal is not to create a real
comparison site for everyone to try and compete to see which AV vendor is better
than the next…
That is not our purpose…
That being said, our purposes in doing AV testing is simple. We wanted to know what
each malware was supposed to be for categorization purposes, and of course just to
see what happened. We collect a lot of malware daily and trying to find ways of
tying our data together is important.
Because we are volunteers and a non-profit we really enjoy sharing what we find
no matter how odd. We even enjoy talking about when we screw something up or
when we encounter something exciting. Everything here is for you our public to
enjoy, discuss, and even criticize…
Shadowserver, 8 September.
