So I just bought a new[to me] Trek 5500 and swapped over the groupset to Sram Rival. The rear derailleur shifts fine through the lowest cogs(12-2?) but it shifts into the second highest(23t) but then will shift right back down. Does anyone know what the problem possibly could be?

did you adjusts the limits and the whole nine yard?

(1) check to make sure the high limit screw isnt preventing the rear der from going far enough.
(2) if it's not, try messing around with the barrel adjusters...2 clicks at a time.

limit screw or cables probably

Sounds to me like you failed to attach the cable with the shifter in the smallest cog position and then make sure that the very first click of the finger lever shifted to the second cog. I believe that if you run out of clicks, the next shift will be back down to a smaller cog. I could be wrong though, since I'm a Campy-only guy and just read about the quirks of SRAM.

Here's what you do:

shift to the smallest cog.
loosen the rear der cable
turn the downtube & derailleur barrel adjusters all the way in, then like 4 clicks out.
go to www.youtube.com/sramtech and watch their "Front and Rear Road Derailleurs" video.
set it up as they tell you, then fine-tune with barrel adjusters

Obligatory Park Tools and Sheldon Brown links.

Follow either of them from the start, w/o skipping any steps.