There are so many gray areas in urban cycling, and this appears to be one of them.

If you had gone into the traffic lane, you would be "that moron who thinks he's a car", and by riding in the crosswalk it's "See, cyclists never follow the rules!"

Just do what feels safe, and don't worry about winning the respect or approval of cagers, because that is a pointless endeavor.