Ohhhh, this stem is indeed going to be a pisser. The crank bolt covers (alloy) are also frozen in place.
I scraped at the rust and there are indeed some spots of indentation as the rust ate material away. The cable guides may be toast, we'll see. I may be able to salvage them with wire brush and rust converter.
Maybe build the areas up with a dab or two of JB weld as opposed to filler.
On the other hand, this frame may be a candidate for a good "drewing".