I travel to DC from the nearby NJ/NY area specifically to ride. I either come in and out for the day, or stay overnight across the Potomac in Alexandria VA.

Yes, DC is a great place to ride; and, no, it's not unsafe. Not only do you have all of the monumental architecture and broad streets of the city, but the various bike trails in and around the city -- the Mount Vernon and W&OD to name two -- are amazing on their own. Georgetown and Alexandria are also good places to ride and explore.