English will get you a long way, and German is good to know as well. German tourists seem to be everywhere in Europe, and a big chunk of western Europe (Austria, Switzerland, parts of northern Italy) is German speaking. When I was cycling in Sicily, a lot of people just assumed that I was German and spoke to me in German.
Greetings and basic phrases will go a long way, people will appreciate your efforts and reciprocate. Carry a phrase book or use the list of phrases in the back of most travel guides, and don't be embarrassed to look through the phrase book before you approach someone or walk into a shop.
Whatever you do, don't just walk up to someone and start speaking English (unless you're in an English-speaking country) as if you expected everyone to learn your language. I used to cringe whenever I saw Americans doing that!