I wouldnt worry about average speed, but if you want to get faster, yes, you will get faster just by riding more if you've just started out. Eventually, you will plateau. If you still want to get faster, you have to push yourself more. Do that a while and you will plateau again. If you continue to want to get faster, you will eventually need a deliberate training program. Eventually, genetics will become your limit and you wont be able to do any better.