Not only all the personal penalties mentioned so far, but folks riding where they shouldn't do a lot of harm to our cause for trail access in the more non-bike friendly parts of the country.
Not only could you get a ticket to spend time with Bubba after being levied a hefty fine, but you could affect a bunch of others in the long run on trail access issues. More people than you could get screwed - pun intended.