Thanks for the responses, BenRidin.
If there is no water jug out front and no sign by the spigot saying it isn't potable, then I would just use the spigot, and not ask questions, especially during a pandemic when they might prefer that people keep their distance.
If there is a water jug, I would just use the water jug, and not worry about whether the reason the jug is out front is the spigot water isn't potable or that they want people to stay away during the pandemic or that they just want be nice and give cyclists especially cold water.