If you're a mountain biker, it's centrally located to give you access to some of the most unbelievable trail riding in - at most - 30 minutes in any direction.
El Cajon an absolute cultural wasteland compared to a city like Philly (sorry El Cajon). It's flat, brown, stucco, and gross. San Diego is also a cultural wasteland, but more along the lines of 'sleepy beach town that really thinks it's a city', and the amazing weather makes up for the serious lack of cosmopolitan feel. It's sort-of big-city feel, but feels less 'dense' and less 'lively' than Philly.
The fixed gear scene is rapidly improving. Lots of cool folks. The Critical Mass group is pretty ******** and unfocused, and does more damage than good, but there's a lot of other rides to go on and people to meet. We also have a velodrome in Balboa Park, if track is your thing.
College bar scene... lots of hot bimbos, frat boys, bleached hair, shallow twits, military meatheads, and brainles surfers. The awesome people are a bit sparse in that realm. Better to meet folks somewhere else, unless one-night hook-ups based completely on bodyfat percentage are your thing.
Expect a horrendous cost of living. The pay scale is about 20% less for the same job compared to San Francisco, and price of median houses is about the same.
You cannot commute on a bike many places. It's fine certain neighborhoods, but they're often joined by large stretches of highway that are either impassable or incredibly dangerous.
All my crankiness aside, I still love it here, and it's a great place to live.