School was problematic for me through grade school and most of high school. I'm introverted, and had poor social skills, and maybe a bit Spectrum-y, and that area has a strong anti-intellectual streak. So, I didn't fit in. And I was one of the smaller kids - second shortest boy in my class in 8th grade! It got to be a lot less of a problem when I shot up to 5'10" by 11th grade. I was a lot less likely to get punched, because I was now bigger than most of my earlier antagonists.

EDIT: So, I didn't enjoy school that much. But I LOVED college.