Thanks for the replies and tips. Already tried the one about resting for a few minutes instead of walking the rest of the way up the hill; It didn't occur to me to even try to ride a little bit more as I was huffing and puffing walking up the hill. I was just focused on making it to the crest before riding again.
Rode in this morning and the only thing I noticed was my butt was sore when I first sat on the saddle. I thought I might be in for an uncomfortable ride but I settled in fine.
I think my plan initially will be to try for 3 days at least so I don't burn out. If I want to do more great, but if I skip a day or two then it's not like I've given up or anything.