My first 1000 mile tour was on a borrowed bike with borrowed gear. The trip cost about $40 and a few favors in 1975. It turns out I loved the activity and slowly, over a decade or so, bought gear and bikes on sale, learned what worked for me and what didn't, made lots of mistakes and spent too much money a few times. Hopefully I'm still learning 40 years into it.
My current set-up, bike plus clothing and camping gear, cost me about $1200. I've ridden quite a few 5000+ mile years with it. You definitely don't have to spend a lot of money to have fun on a bike. For me, that's sort of the whole idea.