I agree. It would look pretty awkward.
But then the thing is, it already looks pretty wierd right now. Because it is the tan version with orange tapes and saddle, but then I have stuck up a black milk crate on my rear rack !!!

Not really to save the money on decent panniers, but I just like the convinience of milk crate for both commuting and utility purposes like groceries and stuff !! So looks is least of my concerns .. Practicality and ease of use is my primary concern ..