Can't really tell by the pic, but there's a good chance that you have a cup & cone bottom bracket, which are quite tinker friendly. sheldonbrown.com, parktool.com and probably bicycletutor.com will tell you all about them. To do a proper job, which would include pulling the crank arms in order to clean, inspect and relube the bearings you need a crank puller. Not that expensive as special tools go, but available in several variations perfect for trapping the unwary buyer.
A "limp-along" bodge job can usually be done with a large plier, a drift punch and a hammer. The drift punch can successfully be replaced with a flat-bladed screwdriver if you don't mind mangling it a bit.

Is this where the bar attaches to the stem?(the bar being the part perdendicular to your direction of travel, and the stem being the part pointing forward.)

If by that you mean that the bar is getting crimped by the clamp I recommend an immediate replacement of the bar. Those "dents" will act as stress risers and can sooner than you think cause the bar to snap off. Maybe not quite as critical on a bike with a fairly upright riding position as on a bike with drop bars, but certainly not a type of injury that I'd like to flirt with.

Don't even try it. Rubber, tape etc is fine for mounting cyclocomputers, smallish lamps and things like that, but it will simply squish for the pressures needed to hold a handlebar in place. If you must, try cutting pieces out if a soda can instead. It's called shimming, and soda cans will work OK for that.

Also, a replacement quill stem isn't expensive. Last one I bought was £12, and for the type you've got I've only ever seen two sizes to choose from. Assuming you can get the diameter of the vertical part figured out you're all set to go shop for a replacement.