I made a longhaul trailer using wheelchair wheels (not completed yet), the hand rims on my wheels can be removed without affecting the structure of the wheel but I just left them on.
Pictures of your wheels/trailer would be good so we could see your setup.
Regards
Tom