you wont, atleast I find myself not needing alot of the speciality tools on a daily basis. I normally dont remove cassettes or bottom brackets (although I bought the tools). I also dont build wheels, once a year (middle of winter) I bring my wheels to my lbs and have them true and tension them (I do own a truing stand). so a basic bicycle building toolset includes:

allen keys
torque wrench (if you have a carbon frame or components)
chain tool
small screw driver
wire cutter
pliers (cables)
hammer (handlebar plugs)
tire levers/pump

thats it.