Where are you starting from, somewhere near SART? Or are you just looking for a mid day ride, not necessarily riding to or fro work?
I personally have never had issues nor been nervous on SGRT and the handful of times I've been on the LART it's been perfectly fine for me too, all the way up to where it crosses over to SGRT near San Gabriel.
Riding along Ocean between LART and SGRT is pretty "meh"... lots of traffic and not a lot of shoulder room, never mind all the traffic lights. SGRT to SART is pretty pleaseant along PCH if you can ignore that niggling suspicion in the back of your head that most of the drivers on Sunset Beach are probably hammered by 11 am.
I agree that getting to PV would be a pain but once you're there, it's got some terrific cycling. The first half of the Tour de Cure stays in PV and it's pretty splendid. It would be worth throwing your bike in the car to drive over every now and then.