I do need to change the habit of watching only the pedestrian's count-down signal without looking at the traffic light.
You certainly do, because you are not a pedestrian and signals that pertain to pedestrians are not your concern other than to prevent you from injuring someone who is walking