You are unfortunately at a "tweener" size so suggestion is go to some shops (like you probably have already done) and test ride some bikes. Sizing will change between style of bike (ie cross country vs. all mountain), size of fork (3, 4 v 5, 6 ") and just generally bike construction. At 5'10" I would think a large, 18.5" would be just right but you never know. I have a bud who is 5'10" who rides a large SC Blur and the bike is really too big for him. He's OK on regular flat trail but down hills, he has trouble reaching off the back of the saddle.

Bottom line it's what feels comfortable for you. Do you have any friends with MTB bikes you can borrow and ride? Really need to get out on a trail to know if the bike will work. Or you can do what I did, bought 4 bikes until I found the right one. Isn't that what we all do?