Yes, changing the stem can affect the handling. Ideally, your hands on the drops should be right over the axle of the front wheel, which will put the hoods slightly forward and the tops slightly behind the wheel axle. If your hands are positioned behind the wheel axle, you'll have squirreliness because you have a bit of wheelbarrowing effect.

You might be better off putting the original stem back on and moving the saddle forward 2cm, or maybe splitting the difference (100mm stem and the saddle 1cm further forward). If these measurements get way out of whack (short stem leading to bad handling plus saddle all the way forward), then it means your frame is too big. Don't forget to recheck your saddle height if you mess with the for/aft position.