The first week is always the toughest on your body and really brings out any little misalignments in your bicycle fit and your riding position. Some aches and pains will lessen and/or disappear as you continue, but those caused by the fit of the bike and your riding position/style should be addressed.
Based on what you wrote:
#1 - You may be pushing too high of a gear - try using a lower gear and spinning faster/easier. Also ensure that your saddle isn't too low. When climbing, don't be too macho to take a break now and then or even walk the bike for a spell.
# 2 & 3 - you may have be supporting too much weight forward on your arms & hands, which can also cause soreness in your neck and shoulders. If the fore/aft position of your saddle is already adjusted properly, don't mess with it, instead you may need to look at getting a different stem that will allow you to have your weight more evenly distributed between your legs, butt and arms. If you have an adjustable stem, tweak its position. Also, if you might want to try different gloves. If you are not using padded gloves, those may help, or, if you are like me, you may need LESS padding in the gloves.
You might also strongly consider taking a day or two off to give your body, knees especially, a little rest. Ice and ibuprofen as mentioned by others.
Be advised that I am not a doctor nor a fit expert, even though I did stay at a holiday inn express last night.
I have attached a word document that has some exerpts from one of the better bike fit outlines I found on the web (with site reference) that may help you adjust the fit of the bike.