The three biggest component upgrades from a functional point of view are double pivot brakes, indexed shifting and brifters. If you already have all of the above then further upgrades will be for show and not significantly affect the performance of the bike. If you're going to buy a group then you should buy it together since (a) the price will be much lower and (b) you can be sure everything will work together. You can get 6700 group sets (8 piece) for just over $800 on eBay (with Bing discount) which includes chain, cassette, brakes, fd, rd, crankset, bbracket and brifters/cables.

Obviously the shifters have to match the cassette for number of speeds and the chain should be the same size or for more gears than the cassette. Derailleurs prefer a chain that is no more than +/- 1 in speeds (i.e. an eight speed chain with 10 speed derailleurs is a bit tight but a nine speed works fine). Some brifters need matching derailleurs because of cable pull (I believe Dura Ace 7700 only works with DA 7700 derailleurs). The latest brake levers have a different pull too so are optimal with matching brakes (i.e. use 6700/7900 brifters with 6700/7900 brakes).

As others have said, a nice set of wheels and tires will get a much more noticeable improvement. A good used set of Mavic Elites can be had for around $300 or new 2009 models on eBay for around $500.