Being thrown into coasting is always scary. Either by forgetting and applying backpressure, or pedalling over a bump and losing your spin and coasting for a little bit. Also for a good while I felt less stable on cornering on my geared bike. The other way happens too, after a long ride on the coastie I started to use the brake on the fixed almost exclusively for a while.

Now that I've been switching between them more often I don't really get surprised by the different drivetrain anymore. I can go for a long geared ride and be completely comfortable coasting and shifting and using only hand brakes to slow down, then I can get on my fixed and skid and skip and stop just fine without touching my front brake.