You can for sure, but should you?
When you say connected I'm assuming you mean the controller is still in the circuit, throttle at zero. In this case you probably will have at least a small current flow due to parasitic losses in the controller, perhaps too a Cycle Analyst running etc.
Battery packs that include BMS may be fooled by this draw and potentially over or under charge the pack.
I always disconnect my pack from the rest of the circuit before charging.
The way I see it, a battery charger is designed to charge a battery, not charge a battery and power up some other devices.
Just my 2 cents.