That may well be true, but... In the past (before my battery died for the millionth time) when I would download my Polar HR data to a scatter-o-gram, it was
extremely rare for me to see any outlying data points. I'd also occasionally look at beat-by-beat tables, and only rarely saw the types of outliers you describe. It may more likely be electrical interference (overhead lines, other people with monitors, etc.).
Do you exercise with others who may be using HRM's? Do you exercise in the area of high tension lines? That, or a defective monitor, is more likely the cause of your problem.