What are outliers and how to detect them in your data
An outlier is a data point that lies abnormally far from the general behavior of a sample. It can be an error, a genuine exception, or a valuable clue, so detecting one is not the same as deleting it. First, you need to investigate.
Imagine a store receives between 20 and 35 orders a day and records 240 on a Tuesday. That 240 stands out, but you don't know why yet. Perhaps there was a brilliant campaign, a duplicate order, or a measurement error. The right decision depends on the cause and the goal of the analysis.
What exactly are outliers?
Outliers are observations that clearly deviate from the pattern followed by the rest of the data. The NIST statistical handbook defines them by their abnormal distance from other values in a sample and warns that what is considered abnormal depends on the context.
An outlier can appear for several reasons:
- An error in entering, importing, or measuring a value.
- A real change in customers, processes, or the market.
- An exceptional event, such as a viral campaign.
- A small or unrepresentative sample.
- The application of a model that does not describe the data well.
That is why there is no universal boundary. A 5,000-euro order might be normal for an industrial company and extraordinary for a small e-commerce business.
Why do outliers matter?
Outliers matter because they can shift averages, distort comparisons, and worsen predictive models. If nine purchases are around 50 euros and one reaches 1,000, the average ticket will rise significantly even though almost no customer spends that amount.
They can also uncover opportunities. A spike in conversions might reveal an especially profitable channel; a sudden drop, a problem with a form; an unusual query, a need that no one was addressing. At Ocelot, we analyze these exceptions within their context before turning a striking figure into a marketing decision.
How is an outlier detected?
An outlier is detected by comparing each observation with the distribution and pattern of the set, first visually and then with an appropriate statistical criterion. No method alone replaces business knowledge.
Charts for locating strange values
A box plot shows the median, quartiles, and outliers. A scatter plot helps to see anomalous relationships between two variables. A time series allows you to distinguish an isolated spike from a sustained change. These are good starting points because they show the shape of the data before applying automatic rules.
Interquartile range rule
The interquartile range, or IQR, is the difference between the third and first quartiles. A common rule marks values lower than Q1 minus 1.5 times the IQR or higher than Q3 plus 1.5 times the IQR as potential outliers. It is useful when the distribution is not perfectly symmetrical and avoids relying too heavily on the mean.
Z-score and robust methods
The Z-score expresses how many standard deviations a value is from the mean. It works best when data approximates a normal distribution, but the mean itself can be dragged by extreme values. The NIST outlier detection guide suggests checking assumptions first and explains robust alternatives based on the median.
Should an outlier always be removed?
No. An outlier should only be removed for a verifiable, documented reason. Deleting it because it is inconvenient can conceal information and produce an artificially tidy result.
A careful approach would be:
- Check the original source and rule out data-entry errors.
- Find out what happened at that time or in that record.
- Repeat the analysis with and without the value.
- Choose a robust method if the value is legitimate.
- Document the decision so it can be reviewed.
A duplicate sale should be corrected. A sale to a real customer should be retained or analysed as a separate segment. If you cannot establish the cause, present both scenarios and explain the uncertainty.
Examples of outliers in marketing and business
In marketing and business, outliers include traffic spikes, unusually high advertising costs, exceptionally large purchases and response times outside the normal range. They can also appear in surveys, demand forecasts and production metrics.
Suppose a campaign receives hundreds of visits from one location without any engagement. That could be automated traffic. By contrast, a page that receives many more enquiries after a media recommendation represents a real exception worth investigating. The numbers may look similar, but the explanation completely changes the action to take.
The key is to connect analytics with context. If you need to turn campaign, website or sales data into understandable decisions, the team at Ocelot can help you separate useful anomalies from noise and build more reliable measurement.
Frequently asked questions about outliers
Do outlier and anomaly mean the same thing?
They are often used interchangeably, although anomaly usually describes a broader unexpected pattern and outlier an extreme observation. The practical distinction depends on the field and the method used.
Is an outlier always an error?
No. It may result from a recording error, but it may also represent a real event or uncommon behaviour. Look for its cause before correcting or removing it.
How many outliers can a dataset contain?
There is no fixed number. It depends on the distribution, sample size and chosen criterion. If there are many, they may represent separate segments or an unsuitable model rather than exceptions.
Which method is best for detecting outliers?
No single method is best for every case. Charts and the interquartile range are useful for exploration, while statistical tests require their assumptions to be checked. Choose according to the type of data and the decision you need to make.
