I've seen it done. Is it a problem?

Have a good lead on a 32H Phil... and have some really nice 24H rims. So, should I just do it? I'm a lightweight. Or, should I pony up and purchase a custom 24H for more coin? Or, eff the whole idea and get 32H rims?