Yes, getting your fit right makes a big difference. And like you say, a lot of getting faster is not getting negative reinforcement for working harder. You can't put it all down to fit, though. Every week you're getting stronger. That'll go on for years.
BTW back in the day I rode rat-trap pedals with clips and straps. I left the right strap just pulled tight, not threaded through the buckle, and only fully buckled the left one. If that'd work for you, that's an immediate and inexpensive improvement you could make.
__________________
Results matter