IMO your seat is a bit too high...There should be a bend in the knees at bottom of the stroke.
Do you warm up your legs at the start of the ride? If not try starting your ride with at least 5 to 10 minutes of easy pedaling then start your plan of the day. Same at the end of the ride.
Start slowly...your body is new to this activity thing from the sounds of it and needs time to adjust.
Have you had a physical? You may want to get a physical just to make sure your body is ok for physical activity.
Take an over the counter aid...Aleve, etc.