I will be receiving a State 4130 single speed bike within the next week hopefully, and I would like to know if a Shimano 105 or an Ultegra caliper brake set can fit this bike.

What you need to know is the reach required. The front is usually not a problem and I'm sure either 105 or Ultegra will be fine. But you'll need to verify the reach of the rear brake and choose a brake accordingly.

By the way, you didn't ask, but if there is a considerable price difference, I'd go with the Shimano 105. As far as braking, they'll be as effective as Ultegra.