I don't use an under-seat bag either.

Short rides (less than 100 miles) - phone goes in the jersey pocket.

Centuries or longer - I put a Bento box on the top tube for the phone, my money clip, and a few energy gels.