If you look at the frame geometry for different sizes, you'll see the top tubes and the seat tubes get longer and longer with larger frames. If you had your saddle adjusted exactly at the same height, the larger the frame, the higher and farther away the headset would be. (I'm saying headset because you have some control on where the stem ends up on the steering tube.) Anyway, that's my thought on it so I think it may explain why a different frame feels different.

I have an idea what you mean by being confused on frame sizes. My inseam was measured at a Trek dealer and they showed me in a book I could take either a 51cm or 54cm Trek 520 (with an inseam of 835mm). Then I took a bunch of measurements as described on the competitive cyclist website. They told me at first I could take 51cm or 54cm then emailed me again telling me the ideal might be around 52cm or 53cm. I currently have a 54cm Nashbar touring frame but when I tried wheels on it with 45mm tires, I didn't have much standover left if I wanted to put my foot down on a steep trail. Since I ordered almost two of everything I needed to buy a bicycle because I kept changing my mind what I wanted on the bike, I ended up choosing to build two bikes so I could try a 52cm as well. Then I might sell the other one. (And I have no idea why I'm rambling, lol, but hopefully someone will get something out of it.)