I was riding a Worksman single-speed bike for a couple of years prior to getting my current bike. That bike was relatively low-geared, which helped on hills. On the other hand, at higher speeds, what works best is giving it two or three fast spins then a quick coast, spin again, then coast. In other words, it teaches you to pedal irregularly. So it's not always better.

I have a friend here that used to ride fixed gear bikes extensively and had to quit due to knee issues- so there can be a downside.