Update: Core workout it helped a lot but one thing that helped tremendously and maybe you can try it yourself is to try to keep your core activated while you ride. I raised my seat because I wasn't confident on certain things but got over that now. But I noticed that if I ride without keeping my tight core I start to get a lower back ache. Once I have it I also notice that when I go back to tightening it, it slowly starts to disappear.

So yeah, thanks again. On the bright side I started p90x2 and the firt phase, half of each week is a core workout so i'll be ripping the benefits hand in hand. So in 90 days you guys shall see my results! XD