However, consider how "survival of the fittest" works -- the fittest survive and pass their genes on to their children. Those that are not fit don't survive, and don't pass on their genes. Civilization, right or wrong, changes that, as what constitutes "fit" is different now, and even those who aren't fit are still likely to survive and reproduce.
Not really. Even if you want to think of it that way, individuals who are equally likely to survive and reproduce are equally fit, by
definition. Anyway, talking about "survival of the fittest" on the level of individual organisms is a poor way to think of it: evolution doesn't act at the level of the individual, but on particular traits as expressed in a population. This makes a big difference.
Just because most individuals with a particular negative trait survive to reproduce does not mean that there is an absence of selective pressure on that trait. Natural selection is exponential, and it only takes a tiny shift in the actuarial odds to create a huge shift when integrated over many generations.
Looking just at intelligence, it seems that studies have shown that
fertility and intelligence has a negative correlation -- in short, the more intelligent people are, the fewer children they tend to have. If this trend continues, it could lead to mankind having a
lower average intelligence over time.
Even if you do accept such a conclusion (I don't), that doesn't point to an absence of evolutionary change. Evolution doesn't care whether or not we get smarter or stupider.