You can do that but the chain is more prone to jumping if you have a cog from a multi-speed set.

Especially if you're like me and eschew tension devices. I swap out cogs, chainrings and halflinks 'til I find a magic ratio on my SS's so run a slightly loose chain at times. I often start with a full cassette to help me determine magic ratio.

For a final product I go with SS-specific chainrings and cogs most of the time. Sometimes I'll run oldschool chainrings, from back in the day when they had no halfteeth.