I have run 700s and 26s on my bike, but my 26s are much fatter than my 700s, so there was no lowering of the bottom bracket. I have to think it would be a noticeable change if you went to a smaller diameter and a narrower tire. Whether it's enough of a change to bother would depend on you. I find the height of my bottom bracket is noticeable primarily when I'm off pavement, and it sounds like your narrower tires would primarily be pavement tires. On the other hand, I haven't found any real benefit in going narrower with my tires. If I don't think I'll be off road, then I just look for a smooth tread, rather than a skinny tire.

The other issue you might face is trying to cover your desired range on one rim. I've run down to 32 and up to 40 on my 19mm wide rims. At the 40 end, it started feeling a little sketchy unless I kept them near the upper end of their pressure range. But I'm a heavy guy, and I'm sure that's a factor. It might not be a factor for you. Still, now that I know that I prefer a wider tire, I look for wider rims to match.

If it were me, and I wanted a narrower road tire for pavement-only rides, I'd build up two sets of rims. 700 rims with an internal width of no more than 19 (slightly narrower if I ever imagined going thinner than 32), and a 650b rim set that was a little wider, somewhere in the 20s, so I would know that I could maximize the tire width to limits of the frame.