Some summers are colder than others. 1996 and 2000, for example, were relatively cool in my neck of the woods. If you are camping at high elevation, it can get cold before dawn. If you get drenched during a storm, you will probably want a way to stay warm at night. Better to carry a sleeping bag and never use it than not to have one when you really need it.
There are featherweight summer bags that will keep you warm down to around 8 degrees C (45 degrees F) and weigh a kilo (2 lbs) or less. If it's made of down, it will pack down real small.