I believe cross levers are designed to give quite a bit of leverage so hand fatigue shouldn't be much of an issue. Road levers give you an extra hand position on the hoods--great for climbing out of the saddle, and for resting your neck and back from using the drops on long level stretches.
I use both brakes with road levers. I did a 250 mile Fleche ride this Spring with a fellow who used the same setup LWaB described. He didn't mention any problems from not having a rear brake.