Chapter 6 “scores” Display
The “scores” display is MultiNav’s main tool for common anomaly detection tasks. This section provides a technical review and demonstration of available parameters. More demonstrations along with detailed explanations are provided in the example sections.
The “scores” display, presents anomaly scores to the analyst for inspection, each anomaly score is displayed in a separate scatter plot. Each point in the scatter plot represents a specific sensor. When the analyst’s hovers over a specific sensor, the matching scores (of the same sensor) from all the other anomaly scores are highlighted (with a larger point and orange color). In addition, the values from the selected sensor are presented layed on top of a functional box plot, which enables to compare the sensor’s readings relative to readings from the other sensors. This display relays on interactivity of linked charts to enable quickly examination of different sensors.
Loading the package and some data.
library(MultiNav)
data<-LambsWeight6.1 Default scores
MultiNav provides several sets of default scores. Details and explanations on each score set can be found in Example I.
6.2 Default univariate scores set
The set of default univariate scores is suitable for initial exploratory data analysis (EDA) and finding univariate anomalies.
Input: The only input required is a continuous multivariate dataset with variables on the same scale. The dataset should be in the format of a data.table, data.frame or matrix. Observations with NA values will be omitted.
MultiNav(data, type = "scores")6.3 Default multivariate scores
6.3.1 “mv” default
# Extracting data from last 3 days
window<-3
data.3D<-data[(dim(data)[2]-window+1):dim(data)[2],]
MultiNav(data.3D,type = "scores", scores = "mv")6.3.2 “T2_Variations” scores
MultiNav(data.3D,type = "scores", scores = "T2_Variations")6.4 Custom scores
MultiNav also allows input of custom scores per sensor.
Input: 1) Raw data, continuous multivariate dataset. 2) Dataset of calculated scores for each sensor (up to four different scores).
quantile_25 <- apply(data,2,quantile,probs=0.25,na.rm = TRUE)
median <- apply(data,2,median,na.rm = TRUE)
quantile_75 <- apply(data,2,quantile,probs=0.75,na.rm = TRUE)
mad<- apply(data,2,mad,na.rm = TRUE)
anomaly.scores<- cbind(id=as.numeric(colnames(data)),
quantile_25,median,quantile_75,mad)
head(anomaly.scores)## id quantile_25 median quantile_75 mad
## 602 602 33.08644 33.96795 41.13540 1.821711
## 608 608 43.84750 51.23000 56.48500 8.925252
## 610 610 34.02000 39.47500 46.03000 8.650971
## 615 615 36.26477 39.83097 44.64344 5.726163
## 617 617 42.03500 48.20000 54.78000 9.377445
## 621 621 43.79750 50.56500 55.00250 9.132816MultiNav(data,type = "scores", scores = anomaly.scores)6.5 Custom quantiles
MultiNav also allows input of custom quantiles, used for creating custom functional box plot envalops (instead of the default).
Input: 1) Raw data, continuous multivariate dataset 2) Pre calculated custom quantiles dataset (in the structure of the example below).
quantiles_matrix<-Calc_quantiles_matrix(data)
head(quantiles_matrix)## seq_id min quantile_a quantile_b median quantile_c quantile_d max
## 1 1 28.62 29.011 34.113 35.837 38.215 40.919 42.900
## 2 2 28.96 29.718 33.523 36.560 38.663 41.079 43.081
## 3 3 29.06 30.154 33.907 36.833 39.015 41.216 43.254
## 4 4 29.05 30.293 34.312 37.121 39.350 41.326 43.418
## 5 5 29.04 30.245 34.616 37.486 39.727 41.888 43.573
## 6 6 29.13 30.187 34.985 37.814 40.030 42.370 43.719MultiNav(data,type = "scores", quantiles=quantiles_matrix)6.6 Examine historical scores
MultiNav also enables to review historical scores per sensor (calculated per “Sliding Temporal Window”). The scores from the last window are presented. When clicking on a point representing a sensor, the display updates. The focus moves to the selected sensor and the sensor’s historical scores are displayed (sliding window scores).
Input: Continuous multivariate data. The window parameter is set. Historical scores per sliding window are calculated.
Note: since scores are calculated for each sensor per each sliding window, this action maybe time consuming!
MultiNav(data,type = "scores", scores = "mv",window=3)There is also an option for providing calculated scores per sensor per window. The calculated scores dataset should be in the specific format as in the example below.
Note: when providing calculated scores per temporal window, the ‘window’ parameter does not need to be set.
scores<-sliding_window_scores(data,14,scores="T2_Variations")
head(scores)## id window T2_mcd50 T2_mcd75 T2_CrouxOllerer T2_SrivastavaDu
## 1 602 1 134.224228 16.172887 13.143718 31.764986
## 2 608 1 7.392593 7.086236 7.338544 3.679985
## 3 610 1 12.618549 13.080255 26.780955 64.062930
## 4 615 1 745.196870 540.777537 14.796287 14.284221
## 5 617 1 7.924094 8.181553 8.188133 4.875164
## 6 621 1 8.754565 6.701607 6.449469 4.858419MultiNav(data,type = "scores", sliding_scores = scores)6.7 Examine outliers
Setting the show_diff = TRUE parameter, the “scores” display adds additional functional box plot of the differenced data.It may provide needed insights regarding the nature of the anomaly.
MultiNav(data, type = "scores", scores = "mv",window=3, show_diff = TRUE)6.8 Functional box plot labels
Setting the parameters xlbl and ylbl will add x and y axis labels for the functional box plot.
MultiNav(data, type = "scores", xlbl = "date id", ylbl = "kg")