













Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
This document, 'Anomaly Detection in R' by Alastair Rushworth on DataCamp, explains the concept of anomalies in data and provides methods for detecting point and collective anomalies using R. the definition of anomalies, point anomalies, collective anomalies, visualizing anomalies with boxplots, Grubbs' test for detecting outliers, and the Seasonal-Hybrid Extreme Studentized Deviate (ESD) algorithm for detecting anomalies in seasonal time series data.
Typology: Lecture notes
1 / 21
This page cannot be seen from the preview
Don't miss anything!
Data Scientist
Defining the term anomaly
Anomaly: a data point or collection of data points that do
not follow the same pattern or have the same structure as
the rest of the data
Visualizing point anomalies with a boxplot
boxplot(temperature, ylab = "Celsius")
Collective anomaly
An anomalous collection of data instances
Unusual when considered together
Example: 10 consecutive high daily temperatures
Data Scientist
Visual assessment is not always reliable!
boxplot(temperature, ylab = "Celsius")
Checking normality with a histogram
Symmetrical & bell shaped?
hist(temperature, breaks = 6)
Running Grubbs' test
Use the grubbs.test() function:
grubbs.test(temperature)
Grubbs test for one outlier data: temp G = 3.07610, U = 0.41065, p-value = 0. alternative hypothesis: highest value 30 is an outlier
Get the row index of an outlier
Location of the maximum
Location of the minimum
which.max(weights)
which.min(temperature)
Monthly revenue data
Grubbs' test not appropriate here
Seasonality may be present
May be multiple anomalies
head(msales)
sales month 1 6.068 1 2 5.966 2 3 6.133 3 4 6.230 4 5 6.407 5 6 6.433 6
Visualizing monthly revenue
plot(sales ~ month, data = msales, type = 'o')
Seasonal-Hybrid ESD algorithm output
sales_ad <- AnomalyDetectionVec(x = msales$sales, period = 12, direction = 'both')
sales_ad$anoms
index anoms 1 14 1. 2 108 2.
Seasonal-Hybrid ESD algorithm plot
AnomalyDetectionVec(x = msales$sales, period = 12, direction = 'both', plot = T)