Thank you everyone for your advice! And a special thanks to @
79pmooney for taking your time writing about
chrondomalacia patella. Even though I'm not 100% sure that might be the case it doesn't sound very unlikely. The symptom you're describing applies very well to my knee and also what you wrote about the temperature. I'm living in Sweden and the day I got the pain it was about 0 degrees celsius (32 F) and on the way home I faced a quite strong headwind. So there's no doubt that my legs was rather cold.
I will try to follow your advice and begin with the stretching exercises you mentioned. The combination of the cold wheather and the fact that I've been very stationary and haven't exercised much this winter probably wasn't very good for my knee...