Sea to Summit Aeros Pillow. If you want to continue rolling up a jacket or filling stuff sack with random detritus from clothing to whatever go for it but I value a good nights sleep. The Aeros pillow takes up very little space and weight and adds a lot of comfort. The only time I might not take it is if I was doing adventure racing and wasn't going to be sleeping much anyway and space was a super duper premium beyond premium but I don't do that or have an interest aside from watching and reading about it so no need to worry.

There should be a little "discomfort" while touring and camping and such like that but very minimal and not to make anyone say this isn't good or I don't wanna do this or I am more of a "insert noun here" because I am "rugged" or whatever. It shouldn't just be a replica of home but it also shouldn't need to be well I don't want to take that because it adds an ounce of weight and takes up a small amount of space. Take it enjoy it and be comfortable and keep your neck in good shape (saying this as I write sitting with my neck in a bad position)