You are not being rude. You are competing against the clock just like everyone else. If you've got the juice left, go for the sprint.

I have passed and been passed many times right before the finish line. Sometimes your sprint will push someone else to pick up the pace and try to catch you.