tsl's post pretty much tells you all you need to know - proper fit, higher cadence, don't worry about progressing too fast. I'd just add that, although a professional fit is an excellent idea, if you're put off by the expense, you can make the most important adjustment - seat height - on your own. The starting point is to adjust the seat so that your leg is fully extended when you put your heel on the pedal, making sure your hips are level on the seat. Ride a mile or two so you know how that feels. Next time you ride, adjust it up or down a half inch or so, and note how that feels. Iterate until you find the sweet spot. You might not find the same spot a professional fitter would, but it'll probably feel a lot better than it does now.

If you don't like having the seat this high because you want to reach the ground easily while seated, realize that bikes just don't fit this way. When stopping and starting, you need to come off the seat. Here's a good summary of this process.