So, quit while you are ahead....sell the frame/fork for $100 and use the parts on a track frame. Get a BikeIsland frame for $110, it is going to be way better than a Fuji, unless it is a top of the line FUji....
Throwing good money after bad is not fun.
ediy: that is a Colnago Sport frame I bought on CL a few years ago. It was made in Mexico for Colnago....but it is a great frame nonetheless.