I work in a bike shop and if you had taken a bike out for a test ride and not come back for a few hours there would be trouble too. We would just call the police and you would be arrested for bike theft.
You have admitted that you did not tell them your plans, so obviously they are upset. A bike that has been taken out for a few hours can have a 100kms put on it. If they let every customer do this the bike can easily need a new chain on it after a month of test rides.
I do not agree with them holding your passport, but the police should have been called, and a "rental" fee for the bike should probably have been paid.