Also note you run the risk of overheating the motor or overloading the battery if you run the motor without a speed controller and have it wired directly to the battery in some way. The controller not only serves to control the speed of the motor, but it also has a maximum current and voltage and prevents the motor from drawing too much current. It all depends on what your motor was rated for and how much current your battery can put out. Unlike computer electronics and other things, motors will pull whatever current they want from the battery depending on the load the motor has to deal with. So if your battery is rated to put out more current then the motor is rated to handle, then the motor could overheat and eventually fry it self. So instead of a burnt out speed controller (which would happen a lot less often if it's all wired up correctly) you got a burnt out motor and that could end up being more expensive to replace then your speed controller!
Of coarse it's possible brushless motors may not draw current in the same way as brushed motors, but if you had a brushless motor, you would need a speed controller for it anyway since brushless motors require some form of electronics to power it correctly since the commutator is electronic instead of being mechanical like a brushed motor.
You could of coarse use a fuse or circuit breaker if you really want to ride without a speed controller. Just know if you hit a hill or wind resistance you may end up replacing the fuse quite often.
I have a brushed motor and while won't last as long as a brushless, it will still go for quite awhile before I would have to worry about replacing the brushes. My speed controller is rather simple in terms of wiring. 3 wires going to the throttle, 2 for the battery and two for the motor. Really the only big change from that of a brushless is that most I know of (for e-bikes anyway) just have a third additional wire for the brushless motor. Some speed controllers may have additional "features" like security key wires, turning signal wires, tail light wires... stuff like that.
If your bike has one of those, you can simplify the wiring quite a bit by simply getting a lower tech speed controller. They aren't really that hard to work with once you make sure the wires have the right connectors on them so that you can service them when needed.