I started my fitness program 4 weeks ago. I'm considerably heavier than you at 266 and 59 yrs. old. I only do 1 timed ride a week. It's 20 mi. and I average between 13/14 mph. I do the same ride for this timed ride and there is one steep hill and one long gradual incline. Riding on strictly flat surfaces I average 15/16 mph. Although I'm younger than you, I'm way out of shape. I suspect the smoking is hurting both your overall speed and your ability to improve those speeds.
You'll notice on this forum there are a lot of successful stories about improving ones fitness. Almost all of them have included some form of lifestyle change. If you're not willing to consider to change your smoking habits, I think you are going to limit your speed and improvement. Not necessarily a wrong choice, but a choice each of us has to make. You seem like a competitive spirit. That spirit may lead you to stop smoking, or you may decide the smoking takes precedent over your competitive nature.
Whatever your choices, keep riding - it will do you good.
John