I can't advocate smaller bags as I have two largish rear panniers on my bike that will sometimes both be full (with a full week's worth of clothes and lunches for a couple of days plus rain gear). However, if you want to keep it lighter, consider what you can leave at work such as your toiletries (if you have a place you can let your towel hang to dry and take it home weekly to wash, there's one less thing to carry, at least on a daily basis).