I forgot to mention after-ride icing and possibly anti-inflammatory meds. When you get home from a ride, try putting ice packs on your knees where they ache (a bag of frozen peas works well; mark and refreeze to use again, but don't eat them once they've been thawed). Leave it on 10 minutes or so. Also, consider anti-inflammatory drugs like aspirin or ibuprofen (not Tylenol, which doesn't have the same effect). Take two when you get back from the ride and maybe two in six hours, or whatever the recommended interval is. They do more than relieve the pain--they reduce the inflammation, too.
Finally, if the pain gets worse with exercise, rather than going away as you warm up, that's bad. Probably ought to see a doctor to avoid possible long-term injury.