I've seen it done before. If it is more comfortable for you, go for it.
But you also might find that the aero position is less comfortable, less stable, and less powerful. It takes time to find the right aero position. Sometimes the frame can help dictate. Sometimes it's just a matter of flexibility.
Ultimately thought, clip-on aerobars aren't a huge expense, so give it a go.