An empirical evaluation about whether or not ML fashions make extra errors when making predictions on outliers

Outliers are people which are very completely different from nearly all of the inhabitants. Historically, amongst practitioners there’s a sure distrust in outliers, for this reason ad-hoc measures reminiscent of eradicating them from the dataset are sometimes adopted.

Nevertheless, when working with actual information, outliers are on the order of enterprise. Generally, they’re much more essential than different observations! Take for example the case of people which are outliers as a result of they’re very high-paying prospects: you don’t wish to discard them, truly, you most likely wish to deal with them with additional care.

An attention-grabbing — and fairly unexplored — facet of outliers is how they work together with ML fashions. My feeling is that information scientists consider that outliers hurt the efficiency of their fashions. However this perception might be primarily based on a preconception greater than on actual proof.

Thus, the query I’ll attempt to reply on this article is the next:

Is an ML mannequin extra more likely to make errors when making predictions on outliers?

Suppose that we’ve got a mannequin that has been educated on these information factors:

We obtain new information factors for which the mannequin ought to make predictions.

Let’s think about two circumstances:

1. the brand new information level is an outlier, i.e. completely different from many of the coaching observations.
2. the brand new information level is “commonplace”, i.e. it lies in an space that’s fairly “dense” of coaching factors.

We want to perceive whether or not, typically, the outlier is tougher to foretell than the usual commentary.