That difference is quite noticeable. Whether or not you need the lower gear depends on you. 36/28 could be ok for randonneuring if the hills aren't too big or you are a strong climber. Most randonneurs I know have even lower gears than 34/28. 34/34 or 34/36 is pretty common for people running a compact crankset. Plenty of people run 30T chainrings or even smaller if they have a triple.