If your cog was intended for use with a 3/32" chain, I don't think it would matter that you're using a 10 speed chain, I think that would work. I think the problem is that your cog is for a 1/8" chain.

Here's a photo of a Surly 3/32" cog with an 11 speed Shimano chain wrapped around it. It would work fine, but I would use an 8 speed chain if this were actually on a bike, if for no other reason than the cost of the chain.