It depends on how concerned you are. In most places if the accessories are fastened on with screws or bolts you can leave them on, if they can easily be removed then take them with you. Use a "U Lock" with a cable. Use the U Lock to lock the rear wheel and frame to the bike rack and put the cable through the front wheel and back to the U Lock. This way the rear rim (the most expensive of the two) is very well secured and as well as the bike as a whole. The cable in the front will likely be enough to discourage someone from stealing it. If you have a quick release seat you will probably want to take that with you.

In some very bad areas people will actually take the wheels with them if they can but if you lock them up properly you should be OK.