You will get more fit. If the "hill" is on your return, that is perfect. Go as fast as you can every trip. You will be a monster soon.
I leave belt and shoes at work with baby wipes and deodorant. I bring Chinos and an oxford shirt with undies each day. I ride in cycling attire. I ride easy on the way in and hard on the way home.
Stick with it.