Okay, I wanted to update this incase someone is doing a search on this noise. I took the BB apart today and found it was a very old sealed unit and isn't a rebuildable one. The guys shop I was working in who has been around these bikes for probably 45 years said, he never saw a sealed unit that old. Anyway, that really wasn't the issue. It was the crank side, which is more in a fixed position was ever so slightly loose. Under load it would move just a bit and when my right pedal was on the downstroke it would make a clunk type noise. He said that is pretty common on the right side. I tightened everything up and no more problems so far. I have a feeling it is fixed for now, but I may need to replace that old BB at some point. I took measurements incase I need to replace. They are 116-117/ 70. I believe that is still available.
Thanks for all the suggestions. Seems the first thing to look for if you have a noise on the downstroke of the crank side is make sure that is tight, which requires taking it apart and tightening it.