There should always be a little slack in the chain, not just when you "pedal fast".
You should be able to easily move the chain up and down at least a 1/2".

A chain that is too tight, will rapidly wear out the chain, cog, and chainring. And that will add up to be a costly replacement.