The star nut would be the issue here, you'd have to file out a large enough hole in it or something, not totally unreasonable though. The brake cable looping back up to the brake might get in the way of spinning too. If you used the spacer idea the brake might be low enough not to hit, but the cable still might.

Seems easy enough to just take the brake off when doing tricks anyway though. That brake lever on the bars would get annoying.