If you let the valve go back to it's natural position in your pic, is that flat puncture at the base of the valve ? If it is, it looks like the valve stem hole on the rim is the culprit and it's either razor sharp or has a bur ? Sand it cleaner and/or smoother. Then you could tape over the hole and then cut a cross patterned slit in the tape over the stem hole. Rather than cut that tape completely out, just put the valve stem thru it and the tape should be between the metal and rubber tube. Might be a tighter fit to get the valve stem thru the rim hole, but at least there's a cushion of tape between the tube and rim hole.

On the Presta valve, are you using the valve nut and how tight is that if you are using it to secure the Presta valve ?