I have both and there are a couple of downsides to rollers to keep in mind that have been mentioned in some of the previous posts.

1) I find using rollers for hard intervals to be challenging. If you push yourself to the point where you feel a bit wobbly at the end of the interval rollers can be problematic. You have to keep pedaling at a reasonable pace to maintain your balance and if you push an interval to exhaustion you can run into problems (loose your balance). Given I have had this experience I tend to hold back a bit on my intervals on rollers for safety sake.
2) Rollers are not great it you want to watch a movie / sports during an extended low to moderate intensity ride. You need to maintain decent concentration while on the rollers (to remain centered) and if you get caught up in a movie or game it's easy to drift off to the side of the rollers and run into problems (fall off).

Ideally having both is the way to go but if I had to choose one I would go with a fluid based trainer for overall versatility.