A new security paradigm for the zero-day advanced persistent threat
What we’ve got
The current security paradigm is derived from Sun Tzu’s Art of War: ‘know your enemy’. The enemy in computer security comprises computer processes: malware. If we know all malware we can stop all malware. That’s the theory. (The enemy in people security – politics – is bad people. If we know all bad people, we can stop all bad people. We’ll come back to that later.)
This is where we have concentrated computer security: knowing threats, learning how to recognise those known threats, and developing methods to stop those known threats. It suffers from two major defects: firstly, we may know what, but we do not know when or where; and secondly, we can never know what we do not know, Rumsfeld’s unknown unknowns, or 0-days.
This explains both the success and failure of our computer security industry: if we spend enough time, money and energy, we can successfully defend against the known threats even if we don’t know when or where they will strike. But we fail against the unknown threats because we have no defence and no defence strategy.
Clearly, the current security paradigm is like the curate’s egg: good in parts. We need a new way of thinking to make the perfect omelette. It doesn’t necessarily mean we need to throw out what we’ve already achieved; it may simply require something extra to deal with the unknowns.
What we need
One characteristic of the current use of these unknowns, the zero-day exploits, is that they are relatively rare and consequently very valuable to the enemy. Because of this they are used sparingly and carefully – they tend not to be thrown away on minor attacks. The zero-day exploit is often used in a serious APT attack; it is used as the doorway for a major covert incursion. But getting in the door doesn’t mean finding the target. There is a time-lapse between breach and success; between getting into the network and actually capturing the flag. This time-lapse is our opportunity to thwart the unknowns. But it all hinges on our ability to rapidly recognise something we’re not even looking for. And that requires a new way of thinking: a new security paradigm.
Think of your network like a spider’s web. You know every µm of thread (cables) and nodes (servers). But more to the point, you know instantly when anything extraneous intrudes on your network. This reverses the current situation: this time you don’t know what, but you do know when and where – and you know it shouldn’t be there. Instant knowledge of what shouldn’t be there, what is anomalous to your true network, is the key to finding 0-day exploits. Achieving this provides the new security paradigm; not merely stopping what we know, but recognising the presence of what we don’t know. It can be done. We know of black holes in space not because we can see them, but because of the effect they have on light passing by them. Their presence creates an anomaly. Everything different has an effect, leaves a shadow, causes a ripple on the surface or vibrates the web. We need to be able to recognise that anomalous effect on our networks – it shouldn’t be there, so it’s likely to be an attack from outside or the ripple of a zero-day inside.
But how can we achieve this? Setting rule-based baselines is not enough. The threat can be above the baseline. Setting normal operating parameters won’t work – the threat can be inside the parameters. Conceptually, our entire network and all of our data has to be the baseline. We need to reduce everything to a norm, so that anything outside of the norm shows up as an anomaly. And that is a Big Data problem beyond the scope of everyday computing. Big Data can be defined as “datasets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time.” [Wikipedia] Or to put it another way, the Big Data problem is that we need to analyse petabytes when we can only handle terabytes: we need to find a needle in a hayfield in the dark.
This requires supercomputer capabilities when we don’t have one. What we do have, however, is lots of smaller computers with lots of spare capacity. Grid computing can provide the super power we need. Which just leaves a need for analytical software able to monitor the entire network dataset and highlight any anomalies as they happen.
One such product was launched this month by Red Lambda: MetaGrid analytics running on the AppIron grid platform. MetaGrid uses its own intelligent algorithm, Neural Foam, to analyse any data it is given. It then reduces that data to a series of visual clusters; from which anomalies can quickly and easily be discerned. It sounds simple, but it obviously isn’t. Red Lambda’s COO Todd Krautkremer expands:
“What MetaGrid does,” he explained, “is collect all available data from both inside and outside the network. It indexes and analyses this information using the technology that we call Neural Foam, which is our proprietary algorithm that does a neural analysis of unstructured data across a dynamic baseline. Neural Foam is all about trying to understand data similarity and association in a dynamic environment. It does this in a manner very different to some other products which try to correlate events. What we do is to cluster events by understanding how that information is related. A lot of this information is shown on screen by a combination of balls and rods. The size of the balls shows the amount of data that is highly inter-related.”
Todd gave me an example of the way in which Big Data is reduced to visually manageable proportions. “We worked with a military/government integrator that’s on the cyber security programme. They collected 350 million lines of data from a supercomputer system that they had access to – they took syslog files representing a year’s worth of events. They ran those events through a SIEM platform and reduced 350 million lines down to 12 million lines. That’s good. But here’s the problem: 12 million lines still far exceeds the ability of a security admin to discern whether or not there’s a threat in there. So we took the same dataset and ran it through Neural Foam on MetaGrid and distilled it down to 47 clusters. A cluster is one of these balls and rods that MetaGrid displays.”
The result is a manageable pictorial view of all the data. As this pictorial view changes, significant differences (anomalies and potential threats) are highlighted. “What showed up on the screen,” he continued, “was a very dense patch – they called it the death star – it was a very dense shape of balls and clusters where the balls were all relatively small but very tightly packed. It was so visually distinct from anything else on the screen that they immediately clicked on it. It just drew their attention. And when they clicked on it, they quickly recognised that there was a steady set of commands from an external address to a major perimeter firewall. Within a couple of minutes of exploring this data they realised that one of their main firewalls had been commandeered by an external party that was sending a regular heartbeat – actually an obscure Cisco command – to determine whether they still had control of the firewall. Finding this heartbeat in 12 million lines of code through traditional methods – well, it simply wouldn’t happen.”
The potential for this approach and its ability to locate the unknown unknowns is clear. The exciting element is that it just gets better. The more data you throw at it over a longer period, the better the system gets to understand the ‘norm’; and the better it gets at highlighting anomalies.
Are there more possibilities, I wondered. Could, when we have learned to trust the cloud, this be offered as a cloud service? “Yes,” said Todd. “Today we can take large datasets and load them into MetaGrid and it will find the anomalies and let a security analyst visualize, explore and search the data to understand impact and root cause. We do this sometimes as part of our sales process. The data could be analyzed in ‘batch’ mode or as ongoing streams. Of course, the trust-factor issues you raised would have to be addressed. Since the AppIron grid is in fact a cloud technology, it can be deployed as a cloud service (think Amazon AWS). However, this is not currently offered by Red Lambda.” Something for the future, perhaps.
What about data sharing? Security analysts are beginning to demand widespread data sharing as a means of levelling the threat landscape with the cybercriminals (see The future of security is analytics and sharing – in conversation with RSA’s Uri Rivner).
“Data sharing is part of our vision and enabled by our technology. A multi-national corporation, for example, could set-up independent grids in business units and then share Neural Foam clustered results (i.e. Foams) across BUs via our federation model. The portable foams can then be correlated against the BU’s clusters. This model can also work across industries and private/public cybersecurity partnerships. Of course, this vision will take some time to realize and will require industries to work more closely together. We are already seeing this happen with certain US federal government agencies.”
The social price
The future is neural, and we should all be a bit nervous: there is no free lunch in this world. The probable price for increased security will eventually be greater state control, and increased loss of privacy.
Let’s start with privacy. Analytics such as MetaGrid look for anomalies based on behaviour. If we’re talking about security, that has to include staff. But MetaGrid eats as much data as it can get. Where do we stop? We obviously monitor staff at work. But the anomalous behaviour that is possibly a spat of spite might arise from a tiff at home. Domestic troubles could cause security breaches – so should we monitor and analyse our staff’s home environment?
And what about state control? We said at the beginning, “The enemy in people security – politics – is bad people. If we know all bad people, we can stop all bad people.” So should the state start monitoring all people in order to locate the anomalous behaviour of bad people? How far do we go to locate potential terrorists; and again, where do we stop? It’s not a problem right now – the necessary computing power is probably still beyond the NSA and MI5, even combined. But compare computing from just 20 years ago with what we have now; and just wonder at what we might have in the future. If the capability is there, the temptation to use it will be irresistible.
So, while data analytics the like of MetaGrid will prove an enormous boon in the fight against cybercrime now, the dangers presented by this new security approach in the future must be tackled sooner rather than later. Personal privacy and state intervention are serious problems that will need to be resolved in this new and evolving security paradigm.