I take my bike on the bus at least five days a week, and it's a Matrix, so it isn't too small even when folded. As everyone else has said, check the regulations of the transit authority in question. In my area bikes are welcome except during rush hour, but folding bikes can always be brought on. I thought a twenty-six inch folder was a nice way to exploit the rule. I always sit in the back, on DART (Dallas) buses there are two benches against the side of the bus and one at the back, forming a kind of "U" and allowing more room for the bike to sit in front of me. I usually don't even fold my bike up unless it is crowded and I want to be out of the way for other passengers. I've never had a bus driver insist I fold the bike, but I usually just use the train for my rush hours travels. I use the velcro strap that came with my Dahon to secure my bike to a support pole on the bus so it can't flop around. I've never had any problems with a driver or passenger. People will constantly ask you were you "get a bike like that." Actually, if I had to point to any problems, it's that people are always asking me about the bike, which is nice except that 50% of them begin the conversation with "How much does a bike like that cost?" I understand they mean no harm, but it's a rude question to ask a stranger--especially when they follow my pat response of "It's a mid-range bike" with "How much is that?" Don't get me wrong, I've still enjoyed talking to most of the people who have started the conversation in the above indecorous manner--they are just overcurious and undermannered but it's still a chance to spread the gospel, if you know what I mean.
I printed up the pertinent section of the DART regulations and keep the page in my tail pack. I've never had a real problem. One time, I started to hop on the bus with my bike unfolded and the driver told me "You know, technically you can't bring that on the bus during rush hours" and motioned me aboard anyway (it was 8:45 and the bus was practically empty). I explained the rule to him, but he still seemed incredulous, so I showed him the printout. It may be that a driver doesn't know the rule because they've never seen someone bring a folder on the bus (or seen a folder period) so the printout is my backup, I don't want to get into an argument with a bus driver.
So check the local rules and enjoy. You are right that the combination of public transport and a folder gives one almost car-like ease of mobility. Dallas is a terrible place for bike commuting and public transport, but I get around just fine using the two combined with living 6 blocks from a train station/bus transit center. Always having the bike allows you to a much wider variety of routes and connections than being on foot.