Looks like your setup should work for ultra-light off-road stuff. Did you cut the handle off your toothbrush yet?
Normally I'd say it is better to keep the weight off your back and put it on the frame, but if you're doing highly technical off-road and need the full suspension, it's a good trade-off.
For road use, I think it'd be usable for short credit-card tours, or very uncomfortable tarp-camping tours.
For longer tours, you really do need more capacity and most people will not be happy with even a well-designed backpack.
The thing I'd be curious about is how substantial weight on the handlebar bag will affect your handling skills and if it will have a critical effect on visibility.