I've seen people do Cat 4/5 races in pro jerseys. At that level, there's a lot of leeway on uniforms. And if you're wearing the jersey of a team in the race that you aren't a member of, how are the officials to know unless someone complains? A plain jersey is the safest approach, but I don't think you need to worry too much about it. Cyclocross example, but I raced the first day of a two-day race weekend once in my old collegiate kit, practically down the street from the university itself. It wasn't a current kit, but still. That was in a Cat 3 race. Maybe that would have been a bigger deal on the road, and I probably wouldn't have done it for a road race or crit, but it's not a big deal, provided you're not doing it to be sneaky, obviously.