You can use the wrong size tube in a pinch. Especially one too small. It will usually work fine. However it often won't hold a patch if you later go to patch it.

Tubes easily expand to fill the area of the tire. So a 700x23 tube will work OK in a 7x32 tire. However patches are not elastic (or not very elastic). So what happens if you patch a 700x23 tube and stick it back in a 700-32 tire is that the tube readily stretches to expand and fill the tire but the patch will not. This puts tremendous sheer stress on the patch adhesive and it will persist as you ride until the patch adhesive fails. The old Rema Tip Top style patch kits that vulcanize onto the tube will probably do best. I'm suspicious about the holding ability of the newer self-adhesive patches.

Bottom line, borrow the wrong size tube for a field repair no problem if it is a new tube but be cautious if the borrowed tube already has patches and don't count on being able to successfully patch it later if it is dramatically smaller than the tire you are riding.