Bottomed out seat tube is generally a bad sign. I'm 6'0" and found a 19.5" Trek to be about right for me, so the 17.5" is probably at your upper end.

When you say the seat tube is bottomed out, if you put your heel on the pedal your leg should be almost completely straight? With a properly fitted bike, you will not be able to put your feet flat on the ground while seated, at best you'll be able to put a toe down.

As for sizing, can you test ride both? If so get the that feels most comfortable while riding. Mountain bike sizing is generally more flexible than road bike sizing, so its likely that you can get a decent fit on 15.5, 16.5 or 17.5. The real question is your personal preference for wheel size, which is a personal feel issue.

The Marlin is a good bike, enjoy whatever size you get!