I had some similar thoughts, and it turned out my friend built just such a bike a few summers ago. He said it was extremely difficult to control, and he barely managed to ride it down the block. Nevertheless he thought that it could be improved a lot by adjusting the geometry.