Keeping it in the basket sounds like a good solution to me.

FWIW I wear a backpack my whole commute which can be anywhere from 25 minutes to an hour long depending on a lot of stuff I won't bore you with.

My backpack is designed for cycling. It's not a "book bag" and it's not intended for hiking. To a certain extent it's just a matter of getting used to it and knowing how to pack and wear it. Most days I barely know it's there.

Typically I carry a smallish laptop, a change of clothes, lunch, and a few odd and ends. I've also used it to pick up some groceries and for other errands. For my needs I prefer it over panniers or a basket but lots of people wouldn't.