I'm pretty sure I just tightened the binder bolt way too much when I originally put it in. Last night I raised it up a bit and noticed that it seemed scored right where the collar was, so I pretty much expected this to happen. Fortunately I had another one in the garage and it broke right outside my house. Unfortunately I was already running late and this made me even later.
Also, it was a cheap ($25) post from Nashbar, so that might have something to do with it. Either way, if you have a carbon fibre seatpost, make sure you don't overtighten the binder bolt!