Try looking at one of the chain cleaners also. Most require no removal of the chain, and they work very well. I have found that the Park Tool brand liquid degreaser works best in chain tools.
If you feel as though the chain's just got to come off to clean it, youi'll need a chain breaker to pop one of the rivets out. The concepts not very hard, but if the chains old or has other problems, it can get tricky to line the pin up correctly and get it back in cleanly. A little practice and it wont be a problem though.
Good Luck.
-Moab