I agree with BikeWNC, it's really a matter of what I had planned for myself that day (which could be thrown out the metaphorical window, of course) but it really comes down to: Is staying in this pace line good for me (for whatever applicable reasons)?
The other obvious reason is if you see unsafe pace-line riding and opt out for your own safety and peace of mind (yes, that happens!).
Rick / OCRR