Stephen Dolan has personally told me that he strongly advises against drilling the F04. I've encountered two people who have tried it and (you guessed it) ruined their forks.
You most likely won't notice a difference. 5mm is pretty slight. If you do buy that Columbus fork and want me to remove the plug for you, let me know.