For those of you who are telling OP to get a power meter- he already has one.
For those of you who are telling OP that he must ride greater volume, I ride around 8hrs/week now. I used to ride 10-15 hrs. What happened? My coach convinced me to ride less. Why? To get faster.
For OP, I ride 2-3 interval sessions per week, plus another 2-3 longer rides. I have abundant rest in my schedule and overall not tons of volume.
It sounds to me like you need a little more of a training plan. Either buy one or better yet consider a great coach. I can give you some tips on finding a good one, should you want to give it a try.