My understanding is that the hash formula used by Avast to store its forum users’ passwords was
$hash = sha1(strtolower($username) . $password);
This is the formula built into the SMF open source forum software used by Avast. It is both good and bad. It confirms that the hash was salted (with the user’s username); but the use of SHA1 will raise some eyebrows. Robin Wood, a professional pentester, suggests that something like Bcrypt would have been much stronger.
Nevertheless, Wood points out, the use of the salt would make cracking much harder. “I have tables for a lot of the common hashing algorithms with just plain words (password, computer etc), but there is no way I can generate them with the salts (kevinpassword, kevincomputer, robinpassword, robincomputer).”
At a pinch, he admitted, he could generate a few tables for the most common passwords, such as root or admin or 123456. So some of the passwords could be cracked relatively easily by a sophisticated hacker, and even more could be cracked if there were world enough and time. Which is actually pretty much what Avast implied in its blog.
Again, my understanding (and you can interpret that any way you like) is that Avast is embarrassed and wishes to do the right thing. It is holding back from making a more complete formal statement simply because it is still investigating the breach — and doesn’t yet know whether it screwed up or was breached by an unknown 0-day in the SMF software.
One thing that does seem clear, however, is that the attackers were indeed sophisticated and not kiddies since the attack coincided with a multi-GBPS DOS attack. Users should therefore assume that if they used a simple password it is now known to the attackers. Those who used a strong password should assume that with time, it will also become known to the attackers. Since Avast immediately took down the forum the passwords will be of little value UNLESS the user reuses the same password elsewhere. It is for this reason that those passwords should be changed immediately; and users who do so should stop reusing passwords immediately.
So what next? In his original post, CEO Vince Steckler said that Avast will rebuild the forum and move it to a different software platform. It will be interesting to see whether the company publicise how it will store its users’ passwords on the new platform.
There are two functions to PR: the first is to shout the good news from the hilltops, while the second is to bury the bad. When bad news hits, PR says very little.
Bad news has hit eBay. It admitted Wednesday that it had been hacked – but it actually gives very little information. This is a mistake. It means that people will comb their words used looking for clues over what has actually happened. The result is conjecture; but what follows is the conjecture of some very clever security people.
Three things leap out from the eBay statement. The first is the repeated use of the word ‘encrypted’, with no mention of hashing for the passwords. The second is the duration of the breach – it occurred in February/March, but was only discovered a couple of weeks ago. And the third is the mention of the database – not part of, nor a geographical region, but the (whole?) database. So what can we surmise from all this?
Firstly, were the passwords encrypted or hashed? It makes a difference. The implication from the statement is that they were encrypted. Most security experts believe that this would be a mistake – passwords should be hashed and salted. In fact, Ian Pratt, co-founder of Bromium, goes so far as to suggest, “It would be rather unusual to encrypt passwords rather than hash them; it’s probably just lack of precision in the statement.”
But that’s what we said about the Adobe breach – and it turned out that the passwords were indeed encrypted rather than hashed. The opinion among the experts I talked to is fairly evenly balanced – while eBay’s semantics suggest they used encryption, many experts find it hard to believe. “This heavily implies that the passwords were not hashed,” said Chris Oakley, principle security consultant at Nettitude. “eBay’s report suggests that the passwords were encrypted rather than hashed,” added Brendan Rizzo, Technical Director EMEA for Voltage Security. Sati Bains, COO of Sestus, said, “Yes… it appears from the comment that they did [encrypt rather than hash].”
“Encryption and hashing are often confused with each other,” explains Jon French, a security analyst at AppRiver. “But from the sounds of the [eBay’s] press release, it seems they were using some sort of encryption.”
Andrey Dulkin, senior director of cyber innovation at CyberArk, is in no doubt. “Indeed, from the eBay statements we understand that the passwords were encrypted, rather than hashed. The fact that the statements repeatedly use the words ‘encrypted’ and ‘decrypted’ supports this interpretation.”
It is, of course, possible that eBay is simply not differentiating between the two processes, since most of its customers will not understand the difference. “The public understand the word ‘encrypted’ more than hashed – so encrypt is frequently used in place of hashed. But it is believed they were hashed,” suggests Guy Bunker, spokesperson for the Jericho Forum and a cyber security expert at Clearswift.
Ilia Kolochenko, founder and CEO of High-Tech Bridge (HTB), doesn’t believe we can tell from eBay’s comments. “The difference isn’t easily understood by users. Even the spokesperson might not be aware. It’s quite possible that the company simply didn’t want to introduce the complexity of describing the technicalities of hashing and salting in a brief announcement.”
What’s the difference, and why does it matter?
The primary operational difference is that encryption can be decrypted; that is the original plaintext can be retrieved from the ciphertext through the use of the encryption key. Hashed outputs cannot be mathematically returned to the original plaintext.
In practice, an entire database of passwords would be encrypted via a single encryption key. But if hashing was used, each individual password would ideally have an unknown value added to it (a ‘salt’) and the results would be separately hashed. “This salt,” explains Voltage’s Rizzo, “is a way to make sure that the hash of a particular password cannot be compared to the known hash of that same password by the attacker through the use of rainbow tables.”
This means that if an encrypted database is stolen, only one key needs to be found to unlock every password in the database.If the passwords are hashed, every single password needs to be cracked individually.
“The advantages to hashing,” Nick Piagentini, senior solutions architect at CloudPassage, told me, “are one, there is no need to manage sensitive encryption keys; two, hashing processes have less overhead to run than encryption processes; and three, there is no need to reconstruct the password data from the hash. Encryption would only be used if there was a need to get the original password back.”
Could the hackers have the encryption key?
This is the 64 million dollar question (and is only relevant if the passwords were encrypted). We don’t know, and we may never know. But it is certainly possible. There are two possibilities: it could have been cracked or it could have been stolen.
Reuters spoke to eBay spokeswoman Amanda Miller:
She said the hackers gained access to 145 million records of which they copied “a large part”. Those records contained passwords as well as email addresses, birth dates, mailing addresses and other personal information, but not financial data such as credit card numbers.
Hackers raid eBay in historic breach, access 145 million records
eBay says the database was compromised some time around late February or early March; but wasn’t discovered until about two weeks ago. What we don’t know is whether the compromise was still in active use by the hackers, what else they did during the two months they were undetected, or whether they left something unwelcome behind. Frankly, I find it hard to believe that having gained access without being discovered the hackers did not have a good look round.
(Incidentally, it is worth pointing out at this point another comment from HTB’s Kolochenko. Basically, eBay’s statement that financial details were safely stored on a separate server is pretty meaningless. “The two servers would have to communicate,” he explained. “The hackers could have installed some malware to listen to the communication between the servers, and sniffed the plaintext traveling between them.”)
So could they have found the encryption key? Opinion is divided. “This is a primary argument for using hashing over encryption for password storage,” comments Nettitude’s Oakley; “an attacker who is able to compromise the database may also be in a position to obtain the encryption key(s).” (Incidentally, if the passwords were hashed rather than encrypted, the hackers could just as likely have found the salt or salt mechanism, rendering the hashed passwords relatively easy to crack via rainbow tables.)
On the other hand, “I would hope they [eBay] didn’t ‘tape the key to the door of the safe’”, comments Trey Ford, global security strategist at Rapid7. “eBay and PayPal have solid security teams, and go through regular third-party assessments. I refuse to believe they would handle encryption key materials that poorly.”
And yet they left the users’ email addresses and other personal information unencrypted. If they were using encryption seriously, they would have used a hardware security module (HSM) to house the keys, and would have encrypted everything. “They do not seem to be very confident about their encryption system,” comments Sebastian Munoz, CEO of REALSEC, “when they are suggesting their customers to reset passwords. If efficiently encrypted, using specific certified hardware, there would be no need to reset the passwords, since protection is guaranteed. When you use a Hardware Security Module (HSM) and not a simple and insecure encryption-by-software process, there is no way that hackers can gain access to the encryption keys.”
Munoz further suspects that software based encryption was used since only the passwords were encrypted. Since software encryption impacts on performance, then cost arguments come into play.
So, given the duration of the breach and the probable lack of an HSM, it is perfectly possible that the hackers also found the encryption key – and if this is the case, they now have access to all of the greater part of 145 million passwords, along with ‘email address, physical address, phone number and date of birth’.
If they did not find the key, would they be able to crack the encryption key? Again, opinion is divided – it all depends upon what encryption algorithm was used. Older encryption algorithms might be susceptible to a ‘known plaintext’ attack (see Wikipedia for details). Getting the necessary plaintext would be no problem. The most popular passwords are remarkable consistent – so a simple analysis with something like DigiNinja’s Pipal on an existing cracked database would provide a fair sampling of plaintext.
“However,” notes Bromium’s Ian Pratt, “assuming any kind of modern encryption (e.g. AES-128) was used then a known plaintext attack should not be feasible to recover the key and hence reveal other passwords.”
“Another approach,” suggested Clearswift’s Bunker, “is to ‘inject’ known passwords (either the hash or the encrypted version) into the database. This would create the equivalent of denial of service for the individual but would allow the attacker free reign over the account.”
The problem is we simply do not know what has happened. eBay’s attempts to downplay the incident is simply leading to conjecture.
While writing this report, Rapid7’s Trey Ford noticed adverts for the sale of eBay’s stolen database beginning to appear on Pastebin. “There has now been a posting on pastebin claiming to offer ‘145 312 663 unique records’ relating to the eBay breach,” he told me by email. We don’t know if they’re genuine, “it’s possible that a criminal has just spotted an opportunity to cash in on the attack with some other credentials dump they have.”
An analysis of the sample provided is inconclusive – the records are possibly genuine but not certainly genuine. But Ford had a look at the sample:
The sample that has been shared indicates that cracking the passwords will take considerable time. This is nothing like what we saw when LinkedIn was breached and the stolen credentials were quickly cracked due to only SHA-1 hashing being used for storage. In contrast, this credentials set is using PBKDF2 (Password-Based Key Derivation Function 2) SHA-256 hashes, which means they employ a strong hash function and also intentionally make cracking them more difficult and slow by individually salting and using a high number of hash iterations. The method used can be regarded as the state-of-the-art way to store passwords on web applications. Again though, we don’t know that these are credentials taken from the eBay breach, and no details have come from eBay on how they secure passwords.
This would fit in with eBay’s apparent confidence that the passwords cannot be hacked. However, Reuters spoke to eBay about the sample, and
eBay’s [spokesperson Amanda] Miller said the information was not authentic.
U.S. states probe eBay cyber attack as customers complain
AppRiver’s Jon French also noticed the Pastebin offer. He told me by email,
I’ll be wary of anything like this until I see people saying they see their own names (or if I end up seeing mine). Eventually if the Pastebin offer is legit, someone will post the file for free somewhere or some security company that buys it will verify authenticity.
His colleague, Troy Gill, a senior security analyst at AppRiver also suggested something that serious criminals will be well aware of: “There is always the remote possibility that this is a honey pot set by authorities to lure in would be buyers.”
eBay is taking the standard route for crisis management: say nothing. This is hugely disrespectful to its customers, who need and have a right to know everything possible. But eBay is also making a mistake in trying to downplay the effect of the stolen data. It says it has “no evidence of the compromise resulting in unauthorized activity for eBay users, and no evidence of any unauthorized access to financial or credit card information.” This is meant to make its customers feel better – the danger is that it might.
What eBay isn’t saying is that the unencrypted personal data also stolen (email address, physical address, phone number and date of birth) is a phisher’s wet dream. Armed with that information criminals will be able to concoct very compelling emails and cold call telephone calls. This is likely to happen on a vast scale and very soon. eBay might feel confidant about its own business, but the data it has lost puts millions of individuals and other companies in danger.
“When companies like eBay keep silent about the details,” commented High-Tech Bridge’s Kolochenko, “I would tend to expect the worst.” It is perhaps worth remembering the Adobe incident, which started off with a breach of a couple of million and slowly escalated into one of the worst breaches in history.
I finally got the email I’ve been waiting for. It’s from Adobe. It starts
As we announced on 3 October 2013, we recently discovered that an attacker illegally entered our network and may have obtained access to your Adobe ID and encrypted password. We currently have no indication that there has been unauthorised activity on your account.
To prevent unauthorised access to your account, we have reset your password…
Let’s have a look at this. “We announced on 3 October 2013, we recently discovered…” What does recently mean? They announced on 3 October not because they had discovered the hack, but because Brian Krebs told the world that he had found stolen Adobe data on the internet. So when it was actually stolen (could have been months earlier) and when Adobe actually became aware of the theft (could have been months earlier) is not known.
Let’s be charitable and say Adobe knew about it by 1 October.
They said that just under 3 million usernames and encrypted passwords may have been stolen. Since I don’t have an Adobe account, and since 3 million is relatively few in the overall scheme of things, I thought no more about it.
A few weeks later Adobe admitted that the true figure is nearer 38 million. That’s getting a bit more worrying, so I checked my browser’s stored passwords and my more recently adopted password manager. Still nothing. No Adobe account. And anyway, Adobe said very clearly that the company had reset all the passwords and notified the 38 million users. I had not been notified. I had nothing to worry about.
But then, about a week later, it emerged that it wasn’t a mere 3 million, nor a more worrying 38 million, but a colossal 150 million. Adobe had notified 38 million out of 150 million – but that is by no means the worst of it. When Paul Ducklin got hold of the database of stolen data, now easily available if you know where to look, a quick analysis showed the user’s email in plaintext, an encrypted password, and the user’s password hint in plaintext.
email addresses – you can infer a lot from an address: usually the user’s name and company. For example, Ken Westin at Tripwire looked through the Adobe hack and found 89,997 military addresses. “This is in addition to the more than 6,000 accounts from defense contractors such as Raytheon, Northrup Gruman [sic], General Dynamics and BAE Systems we also found,” he wrote. “Also, on the federal side, there were 433 FBI accounts, 82 NSA accounts and 5,000 NASA accounts.” So, choose your company, guess the user’s name, look through LinkedIn and Facebook and you’ve got enough for a pretty compelling targeted phishing attack.
encrypted passwords – passwords should be hashed and salted with a slow hashing algorithm; they should not be encrypted. Hashing means 150 million passwords need to be cracked; encryption means that one key needs to be cracked and all 150 million passwords are known.
password hints in plaintext – oh, really! Why bother cracking the passwords when the hint will let you guess it? What do you think is the password when the hint is ‘57’; or ‘the bad disciple’?
So Adobe really cocked-up. They didn’t protect the data, they didn’t store it correctly, and they tried to minimise the extent of the damage. And still it gets worse; because they tried to suggest, don’t worry, most of these accounts aren’t real, they belong to people who just signed up to get promotions or freebies.
Here’s the real danger. In that great mass of one-off freebie-chasing accounts numbering anything between 38 million and 150 million are people who signed up, used a password that they can’t remember, and are completely unaware that their password is now compromised. What if these people signed up years ago before password thefts became a dime a dozen, and lazily used the same password as they use on their email address? There is no way that they can retrieve that password. They now have no way of knowing whether any or which or all of their other accounts have been compromised by Adobe’s failure to adequately protect this password.
One final point. I said at the beginning that I had been expecting the email from Adobe. That’s because I checked with LastPass (who has a little routine that will tell you whether you’re included in the hacked data) and learnt that although I couldn’t ever remember creating an Adobe account, at some point I must have done, because there I was.
So, at least six weeks after it knew of the breach, Adobe bothers to tell me that someone “may have obtained access to [my] Adobe ID and encrypted password” when the world and his dog has access to that encrypted password. I know; Ken Westin, Brian Krebs and Paul Ducklin almost certainly know; LastPass and the hackers most definitely know; and anyone who cares to look will also know. Adobe, however, doesn’t know and continues to insist that ‘an attacker… may have obtained access.”
How dare they, after all this time and all these mistakes, still try to save face at my expense?
What is a hack? No, seriously, I need to know.
Last weekend the People/Mirror reported that Scout7 had been hacked and Manchester City’s scouting database compromised.
Scout7 came back and said it hadn’t been hacked and the integrity of its systems was sound. But City’s database was accessed by someone other than City.
Scout7 was saying that as far as its systems were concerned, it was a legal access via genuine credentials — implying that City must have lost, mislaid, or had its password stolen. It’s an interesting idea. The implication is that if you lose your house-keys and someone finds them, gets in while you’re out, and reads your personal, private diary, you haven’t been burgled.
That, of course, is emotionally absurd. But Scout7 is saying that it (the housebuilder) cannot be blamed for the burglary and doesn’t need to do anything about it. We’ll come back to that.
Meantime, how does this apply to ‘breach notification’? Is a breach a hack? Is the illegal use of legal credentials by a clear bad guy something that will require notification? Will companies be able to claim, we weren’t breached because the hackers got in through legitimate passwords, therefore we don’t need to tell anyone?
Incidentally, Kurt Wismer has an interesting story equally hinging on lack of semantic clarity: was the poor targeting in Stuxnet down to some lax manager saying , ‘make me a virus’, when he really meant, ‘make me a trojan’? Worth reading.
But back to Scout7. No, it cannot avoid its liability by implying it was a customer’s fault for losing his/her password. We all know that passwords do not provide adequate access security. So relying on them, and not adding a second factor to the access control, is effectively building something not fit for purpose. So as far as I am concerned, it got hacked.
The Data Protection Regulation should be amended to force companies to disclose how passwords are stored
Over the last couple of days it has been disclosed that an amazing amount of personal data on 1.1 million Americans has been lifted from the US Nationwide insurance group. Passwords do not appear to be involved – it’s a storage of data rather than an interactive site. But the point is that this data would appear to have been unencrypted – at least the company concerned hasn’t specified one way or the other; and that’s the problem.
Time and again we learn of plaintext passwords being stolen. Plaintext is unacceptable, but it happens. Sometimes, they are stored hashed by SHA1. This is unacceptable because dictionary attacks and Jens Steube’s newly announced brute force attack makes them surprisingly vulnerable; but it happens. At the very least, passwords should be stored hashed with SHA1 – preferably better – and salted.
I for one would be reluctant to commit my password to any site that stores that password with anything less than salted SHA2. But they don’t tell us, do they.
So I call now for the European Commission to amend the proposed Data Protection Regulation to include a requirement for all sites that store user passwords to make it clear on their site, at registration, precisely how those passwords are stored: plaintext, hashed (with what), or hashed and salted. This is the only way we will be able to force vendors to improve the way in which they handle our data.