Have you ever checked out the Haro line of mountain bikes. They tend toward very low standover heights and relatively long top tubes. I suspect that it is because of their background in BMX and dirt jumping.

I have a V4 frame with a 23+ top tube that has very little room for a water bottle on the seat tube because the front diamond is so compressed. (extreme compact top tube)

The complete bike with disc's and a Marzocchi EXR fork (OEM version of the MX comp with air adjust) comes in at just under 25lbs. This setup would shave hundreds off the cost of the more expensive frame and is very fine for just the type of riding you like. You will also have the option to put more agressive tires on it and gear it up for some rougher trails should you want to. (having 2 sets of wheel/tire/gears transforms the beast from rail to trail in minutes.)