My colleague Coda Hale just released a sweet bcrypt-ruby gem that does password hashing right. It also provides for future-proofing by enabling you to assign a computational cost for generating a password hash and letting you version your password hashes. You have no more excuses.
The idea is simple: don't have any direct links in your database between your users' "public" data and their private data. Instead of linking tables directly via a foreign key, use a cryptographic hash [1] that is based on at least one piece of data that only the user knows—such as their password. The user's private data can be looked up when the user logs in, but otherwise it is completely anonymous. Let's go through a simple example.
Let's say we're designing an application that lets members keep a list of their deepest, darkest secrets. We need a database with at least two tables: 'users' and 'secrets'. The first pass database model looks like this:
The problem with this schema is that anyone with access to the database can easily find out all the secrets of a given user. With one small change, however, we can make this extremely difficult, if not impossible:
The special sauce is the 'secret_key', which is nothing more than a cryptographic hash of the user's username and their password [2]. When the user logs in, we can generate the hash and store it in the session [3]. Whenever we need to query the user's secrets, we use that key to look them up instead of the user id. Now, if some baddie gets ahold of the database, they will still be able to read everyone's secrets, but they won't know which secret belongs to which user, and there's no way to look up the secrets of a given user.
Update: A commenter on the Wesabe blog brought up the important point of what you do if the user forgets their password. The recovery method we came up with was to store a copy of their secret key, encrypted with the answers to their security questions (which aren't stored anywhere in our database, of course). Assuming that the user hasn't forgotten those as well, you can easily find their account data and "move it over" when they reset their password (don't forget to update the encrypted secret key); if they do forget them, well, there's a problem.
I mentioned earlier that you store the secret key in the user's session. If you're storing your session data in the database and your db is hacked, any users that are logged in (or whose sessions haven't yet be deleted) can be compromised. The same is true if sessions are stored on the filesystem. Keeping session data in memory is better, although it is still hackable (the swapfile is one obvious target). However you're storing your session data, keeping your sessions reasonably short and deleting them when they expire is wise. You could also store the secret key separately in a cookie on the user's computer, although then you'd better make damn sure you don't have any cross-site scripting (XSS) vulnerabilities that would allow a hacker to harvest your user's cookies.
Other holes can be found if your system is sufficiently complex and an attacker can find a path from User to Secret through other tables in the database, so it's important to trace out those paths and make sure that the secret key is used somewhere in each chain.
A harder problem to solve is when the secrets themselves may contain enough information to identify the user, and with the above scheme, if one secret is traced back to a user, all of that user's secrets are compromised. It might not be possible or practical to scrub or encrypt the data, but you can limit the damage of a secret being compromised. My colleague and security guru Sam Quiqley suggests the following as an extra layer of security: add a counter to the data being hashed to generate the secret key:
secret key 1 = Hash(salt + password + '1')
secret key 2 = Hash(salt + password + '2')
...
secret key n = Hash(salt + password + '<n>')Getting a list of all the secrets for a given user when they log in is going to be a lot less efficient, of course; you have to keep generating hashes and doing queries until no secret with that hash is found, and deleting secrets may require special handling. But it may be a small price to pay for the extra privacy.
Finally, log files can be a gold mine for attackers. There's a very good chance you're logging queries, debug statements, or exception reports that link users to their keys or directly to their secrets. You should scrub any identifying information before it gets written to the log file.
The privacy wall is far from a silver bullet. Privacy and security are hard—really hard—particularly so if your app is taking private data and extracting information out of it for public consumption, like we are at Wesabe. The privacy wall is one of a number of methods we're using to insure that our users' private data stays that way. If you're lucky enough to be going to ETech next month, definitely check out Marc's session on Super Ninja Privacy Techniques for Web App Developers.
I hope you found this helpful. Let me know what you think; I appreciate any and all feedback. And if you've got any cool privacy techniques up your sleeve, share the knowledge!
<p>[1] A cryptographic hash is way of mapping any amount of plain text to a fixed-length “fingerprint” such that the same text always maps to the same hash, and given a hash, it is impossible to generate the text from which it was derived. Hashes are wonderful things with many uses. If you’re a developer, and you didn’t already know this, stop reading now and go here or here, and learn how to generate a SHA1/2 hash in your programming language of choice. Come back when you’re ready. I’ll wait.</p>
<p>[2] You can throw in a salt too, to be safe; just make sure that you’re not using the same hash that you’re using for checking the user’s password. You are smart enough not to store passwords in plaintext in the database, aren’t you?</p>
We live in Campfire at Wesabe, so naturally we've developed a number of tools and hacks around it. One of these days I'll get around to writing about the Campfire bot framework we've developed, but right now I'll start with something simpler: a way to upload screenshots to Campfire with only a few keystrokes (riffing of my colleague Coda Hale's desire to eliminate the need for a mouse). Note that this is for Mac users only, so if you're one of the poor souls who hasn't seen the light yet, you can stop reading now.
Two prerequisites (other than a Mac): Quicksilver and Pyro. Quicksilver should be familiar to most Mac users already. It is an incredibly powerful application that can be quite obtuse but can save you gobs of time if you learn how to use it. Pyro is a client for Campfire. Why do you need a client for a browser-based chat app? Getting message notifications in your dock is one reason, but the biggest reason is that there seems to be a memory leak in Firefox that causes it to grind to a halt if you've had Campfire up for too long. But I digress.
You first need to enable the Screen Capture Actions plugin in Quicksilver (go to Plugins -> Recommended, and check Screen Capture Actions). Then to go Catalog -> Quicksilver and make sure that Internal Commands is checked). This should give you Capture Region, Window, and Screen commands.
Next, the secret sauce: a bit of AppleScript that uploads a file to Campfire via Pyro:
on open theFile
tell application "Pyro"
upload theFile to room "[your campfire room name]" in campfire "[your campfire].campfirenow.com"
end tell
end openPaste that into Script Editor and save it as PyroUpload in ~/Library/Application Support/Quicksilver/Actions (if this folder doesn't exist, create it), and restart Quicksilver.
Now you can upload a screenshot with just Cmd - Space - [Capture Window|Region|Screen] - Return - [take your screenshot] - PY - Return.
Have fun!
I’ve been too tired today to do any actual work, so I’ve spent much of the day camping out on the Wesabe site stats. It’s terribly exciting having so many people hitting your site and so many signing up (almost 1/3 of our unique visitors have created accounts). The site has been humming along beautifully, too–major props goes out my colleague Coda Hale for his Apache/Mongrel/Pen prowess.
Anyway, I just wanted to share one of the more interesting stats from our analytics (Mint–very tasty):
That’s a lovely thing to see. Granted, these are largely very tech-savvy, early-adopter people at this stage, but it’s heartening to see IE getting the beatdown.
The reason I’ve had my head in the sand for the last six months is that I’ve been working on a new startup, and we’ve finally launched publicly. Check us out. Here’s a screencast demo of the service:
Let me know what you think. (Or better yet, send your thoughts to support@wesabe.com.)