Sound like me when I got back into cycling a couple years ago.
At first it was all I could do to ride around the block. It was a gradual process but I just kept at it gradually increasing my distance and exertion level. It's taken a while but now I'm a bicycling god!
(...or at least a helluvalot better than I used to be.)
You're definitely using different muscles than you're used to and it takes a while to develop them and all the other physiological stuff to support them.
So it sounds to me like you just need to stick with it and have some patience.