See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/335339292
Support Vector Machines for Anomaly Detection from Measurement Histories
Conference Paper · July 2012
CITATION
1
READS
24
2 authors:
Some of the authors of this publication are also working on these related projects:
International Symposium on Structural Integrity of Old Steel Bridges (ISSI-Bridges 2017) View project
Fatigue fracture of old metallic structures (i.e. bridges, rails etc.) manufactured at the turn of the 19th and 20th century View project Rolands Kromanis University of Twente 21 PUBLICATIONS 140 CITATIONS SEE PROFILE Prakash Kripakaran University of Exeter 46 PUBLICATIONS 552 CITATIONS SEE PROFILE
All content following this page was uploaded by Rolands Kromanis on 23 August 2019.
1
Support Vector Machines for Anomaly Detection from
Measurement Histories
Rolands Kromanis*, Prakash Kripakaran
College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK *E-mail: rk296@exeter.ac.uk
Abstract. This research focuses on the analysis of measurements from distributed sensing of structures. The premise is that ambient temperature variations, and hence the temperature distribution across the structure, have a strong correlation with structural response and that this relationship could be exploited for anomaly detection. Specifically, this research first investigates whether support vector machines (SVMs) could be trained to predict structural response (strains) from distributed temperature measurements and subsequently, if this could be employed in an approach for anomaly detection. To test this hypothesis, a SVM is first developed to model the relationship between strain and temperature measurement histories from a numerical model of a bridge girder. It is then evaluated for damage detection by comparing its strain predictions with simulated measurements of the bridge in a damaged condition. Results show that SVMs that predict structural response (strain) from distributed temperature measurements could form the basis for a reliable anomaly detection methodology.
1. Introduction
Bridges are valuable assets of the national highway infrastructure and their maintenance and management imposes a significant cost on the economy. For instance, local authorities and Network Rail in the UK (Cole, 2008) estimated that they would require over £1.95 billion for the repair and strengthening of their bridge stock. Bridge engineers have therefore expressed an interest in novel technologies and approaches that reduce maintenance and management costs. Current bridge management systems rely primarily on visual inspections, which often fail to detect early-stage damage (Brownjohn, 2007). Remedial repairs are thus taken at an advanced stage of deterioration; this can be expensive and also cause significant traffic disruption. Sensor systems have the potential to support detection of the onset of damage and could thereby enable preventive maintenance and repair, leading to cost savings in long-term management.
In the last decade, Structural Health Monitoring (SHM) systems have been deployed frequently on bridges to continuously record structural response and ambient environmental conditions (Brownjohn, 2007). For example, three long-span bridges (Ni et al., 2008) – Tsing Ma bridge, Kap Shui Mun bridge and Ting Kau bridge, are continuously monitored using over 800 permanently-installed sensors as part of the Wind and Structural Health Monitoring System (WASHMS) by the highways department in Hong Kong. These systems measure several structural and environmental parameters on a daily basis and, in the process, accumulate a large database of measurements. However, reliable methods that support timely and meaningful interpretation of measurement time histories are still to be developed. Data-driven methods that rely on minimal structural information offer a lot of promise in this direction.
Previous research (Ni et al., 2005) in SHM has shown that ambient temperature changes have a significant influence on structural response. Daily and seasonal temperature variations are known to cause deflections and deformations that are comparable to service loads on bridges (Moorty and Roeder, 1992). Visual examination and analysis of measurement histories have
2
suggested that ambient temperatures are correlated with structural response (Mirambell and Aguado, 1990; Moorty and Roeder, 1992; Yuen and Kuok, 2010). Posenato et al. (2010) used correlations between strain measurements due to seasonal temperature variations for anomaly detection. However, their approach based on moving principal component analysis (MPCA) requires a large set of reference measurements and is also unable to detect anomalous behaviour unless there is significant damage. Laory et al. (2011) showed that removal of seasonal temperature variations from measurement histories could detrimentally affect the performance of MPCA. Previous studies have not attempted to use the relationship between temperature variations and response measurements for anomaly detection.
This research aims to develop a faster and more robust method for anomaly detection by taking advantage of the correlation between temperature distributions across a structure and the measured structural response. It attempts to explicitly capture the relationship between temperature distributions and strains using support vector machines (SVMs), and exploit this relationship for damage detection. The study examines whether a SVM would be able to accurately model the relationship between strains and ambient temperatures from reference measurements and thereby detect any deviation from normal behaviour. Daily temperature measurements from the European Climate Assessment & Dataset project (ECAD) (Klein Tank et al., 2002) are used for simulation purposes. Strain histories are obtained from simulations of a numerical model that represents a bridge in healthy and damaged states.
2. Numerical Model
We first describe the numerical model that forms the testbed for the anomaly detection methodology. A model of a reinforced concrete girder is created using 8-node plane stress elements in Ansys (2011). Each element has the following dimension: 360 mm ×300 mm × 500 mm (ℎ × ℎ × ℎ ). The girder replicates a typical beam from an integral bridge, with both ends fixed and a roller support in the middle to simulate a pier (see Figure 1).
Fiber Bragg grating (FBG) sensors that measure both strains and temperatures are assumed to be present on top and bottom faces at the quarter-spans of the girder. They have accuracies of ±1μɛ and ±0.1°C. The locations of these sensors are shown in Figure 1 as S-1, S-2, etc.
Figure 1: Numerical model of a bridge girder with S-i (i = 1, 2, …, 13) showing the assumed FBG sensor locations.
The main purpose of setting up the numerical model is to simulate measurements of strains and temperatures due to the daily and seasonal temperature variations. The temperature distribution in a bridge is dependent on several factors including the ambient temperature, the geographical orientation of the bridge and its exposure to the sun. These effects could lead to various types of temperature gradients in the bridge. In this study, the following three types of temperature distribution (see Figure 2) are modeled:
3
(i) TEMP1: This scenario corresponds to a linear temperature gradient across the depth of the girder (Figure 2a). This is similar to those used in previous studies by Posenato et al (2010).
(ii) TEMP2: This scenario models a linear temperature gradient across the depth of the girder, and a linear temperature gradient along the length of the right-hand span of the girder (Figure 2b).
(iii) TEMP3: This scenario simulates linear temperature gradients across the length and depth of the girder (Figure 2c).
Figure 2: Three temperature distribution scenarios are shown above; arrows indicate the direction of temperature increase.
The temperature histories from the ECAD project (Klein Tank et al., 2002) are used to define the temperature distributions outlined in Figure 2. The histories are comprised of minimum, average and maximum daily temperature readings for a specific geographic location. Values for T1-T6 in Figure 2 for each time step are derived from the ECAD temperature histories. This study uses temperature measurements for Camborne, Cornwall, UK.
Sensor readings are assumed to be taken during the hours when the bridge has minimal vehicular traffic such as at midnight. This is to ensure that the effects of ambient temperature variations dominate the measurements. This study also assumes that the monitoring system collects only one reading per day. Three loading situations are considered:
(i) L1 → temperature loading only with no traffic loads;
(ii) L2 → Temperature loading (L1) + light vehicle load (e.g. cars);
(iii) L3 → Temperature loading (L1) and light or heavy vehicle load (e.g. trucks).
The probability that a light vehicle is crossing the bridge during measurement collection is set as 0.2 or two out of ten measurements may be collected while a light vehicle is crossing the bridge. Similarly the probability of a heavy vehicle crossing the bridge is set as 0.05. The traffic loads are modeled in term of equivalent axle loads and the model is setup such that the location of the vehicle is randomly chosen during each measurement time-step. Light vehicles could have axle loads of up to 10kN, while heavy vehicles could lead to axle loads of up to 62.5kN.
The model is used to simulate measurements from a bridge in both normal and damaged states. The structure is assumed to be in normal condition for 1 year. Damage is introduced after 1100 days as a reduction in the material stiffness of the elements indicated in Figure 1. The following damage cases are considered:
(i) D1 – instant stiffness loss of 50%; (ii) D2 – instant stiffness loss of 30%;
4
3. Support Vector Machines
SVMs are supervised learning methods that are widely used in the computing community for prediction and classification tasks in a range of domains such as image processing and network security. While numerous other classification algorithms such as K-means clustering and artificial neural networks (ANN) are available, SVMs are chosen in this research due to their many successful applications for anomaly detection in diverse subjects such as computer networks, finance and medicine (Chandola et al., 2009, Heller et al., 2003). SVMs have also been previously used in SHM applications. Shengchao et al. (2011) proposed a support vector regression (SVR) based fault detection method to detect anomalies in the structure of F-16 fighters without requiring prior measurements in a faulty condition. Other applications in SHM include structural integrity assessment (Noori et al., 2010) and structural system identification (Saitta et al., 2010). They have been shown to effectively capture correlations between temperatures and modal frequencies (Ni et al., 2005). However, previous studies have not examined the application of SVM for quasi-static measurements, the focus of this research.
In conventional SVM, the datasets are often first transformed to a higher dimensional feature space using a kernel trick. Optimization is used to find the hyper-plane that best separates the datasets in this transformed feature space. Support vectors refer to the vectors that define the hyper-plane. The process of finding the support vectors can be computation-intensive due to the tuning required as well as the quadratic optimization that is involved. In this study, however, SVM is used in the regression mode since measurements taken from the structure in a damaged state are unlikely to be available for training purposes in practice and furthermore, since simulating all possible damage scenarios is practically infeasible.
3.1 Support Vector Regression (SVR)
SVR uses the same features that form the basis of SVMs. The only addition is a loss function that determines the degree of complexity and generalization provided by the regression. There are two main classes of SVR approaches – ɛ-SVR and ν-SVR. ν–SVR is used in this study since it requires less tuning and fewer number of parameters than ɛ-SVR. It automatically minimizes the loss function and has been shown to support more meaningful data interpretation (Schölkopf et al., 2001, Chang and Lin, 2002); this premise is also validated by results from this research.
As for any machine learning technique, the core task in developing a regression model is to find model parameters that minimize the prediction error. The sensitivity of the SVR model is greatly dependent on the value specified for ν – a parameter which determines the number of support vectors and the number of bias support vectors. In addition to ν, values for two other parameters – a regularization constant (C) and gamma (γ), that also affect the performance of the SVR model have to be specified. Five-fold cross-validation is employed to evaluate the best values for C and γ. In this procedure, a data set is split into five equal parts such that one part constitutes the learning set that is trained on the other four parts. The values for C and γ are chosen such that they maximize the coefficient of determination (or squared correlation coefficient (R2)), which is derived as follows:
ଶ = 1 −∑
−
ଶ
ୀଵ /∑ ୀଵ −ଶ, = 1, 2, … , (1)
where ypi and yri represent the prediction and measurement at the ith time-step, is the mean
5
functions were considered in this research. However, for reasons of brevity, results are presented only for linear kernels.
3.2 SVR for Damage Detection
Strain predictions from the model described in Section 2 are taken as measured strain histories for this study. The response generated for an initial reference period (one year) without any damage to the structure constitutes the training set. Increasing levels of damage are introduced in the numerical models to generate the test sets. A SVR model is created for each strain measurement location. Distributed temperature measurements constitute the input to the SVR model. The difference (∆yi) between the predicted and measured strain for a given
sensor location (Eq. 2) is a measure of the structure’s deviation from normal behaviour. Δ = −, = 1, 2, … , (2)
4. Results
The efficiency of proposed algorithms is evaluated on data from simulations. The study assumes that measurements are free of errors. Measurements could be simulated using the numerical model for several scenarios, where each scenario is a combination of a temperature distribution, a loading situation and a damage case. For example, scenario TEMP1L1D1 refers to measurements simulated from the numerical model using the temperature distribution TEMP1, the loading case L1 and the damage case D1. Figure 3 shows the strain history at sensor S-7 of the girder (Figure 1) for this scenario. The figure shows that damage modeled as a 50% loss in stiffness is not visually discernible from the strain time history. The effects of damage are masked by the large changes in strains due to daily and seasonal temperature changes.
A SVR model is first trained from measurements taken when the bridge girder is in normal condition. For this example, measurements taken during the first year constitute the training set. The Libsvm package (Chang and Lin, 2011) is used for creating a SVR model. A linear kernel is chosen to predict strain values. Predictions from a SVR model for the previously described scenario (TEMP1L1D1) are illustrated in Figure 4. The regression model predicts strain measurements with a very high degree of accuracy.
Figure 3: Temperature (left) and strain (right) readings from sensor S-7.
0 500 1000 1500 2000 -5 0 5 10 15 20 25 30 T e m p e ra tu re ( oC ) Readings 0 500 1000 1500 2000 -100 -90 -80 -70 -60 -50 -40 -30 -20 -10 0 S tr a in (εεεε ) Readings Damage
6
Figure 4: Comparison of measured and predicted strains for two and a half years (left); and an enlarged view of the comparison over one month (right).
The SVR model is subsequently used to classify whether a new set of measurements represents an anomaly. The difference (∆y) between the strain prediction from the model and
the measured strain could be an indicator of damage. This difference is plotted for sensor S-3 in Figure 5. Significant signal oscillations in ∆y, which correspond to seasonal temperature variations, are observed after damage is introduced (see Figure 5). This illustrates that, after damage, the difference between predicted and measured thermal response is amplified in proportion to the ambient temperature. Therefore this time history of differences could be analyzed by signal processing methods such as fast Fourier transform (FFT) or principal component analysis (PCA) to automatically detect anomalies.
This research applies moving fast Fourier transform (MFFT) (Sherlock and Monro, 1992) and MPCA to find statistical evidence of anomalous behaviour from the time series of prediction errors produced by a SVR model. An anomaly is said to be detected when the respective indicator, which is the amplitude of the lowest frequency in case of MFFT and the principal eigenvector in case of MPCA, exceed specified threshold limits. The time to damage detection is measured as the number of days between the introduction of damage and the detection of an anomaly. MFFT evaluates the fast Fourier transform of a window of data points from the time series, which in this case corresponds to a time series of ∆y values. Significant jumps in the amplitude of the smallest frequency from MFFT can be classified as an anomaly. An illustration of anomaly detection using MFFT is shown in Figure 5. Thresholds for anomaly detection are determined as six times the standard deviation of the signal.
Figure 5: Time series of the differences (∆y) between measured and predicted strain values at S-3 from a SVR model for scenario TEMP1L1D1 (left); Results from MFFT of the ∆y time series (right).
0 100 200 300 400 500 600 700 800 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Readings S tr a in (εεεε s c a le d ) monitored predicted 120 125 130 135 140 145 150 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 Readings S tr a in (εεεε s c a le d ) monitored predicted 0 500 1000 1500 -0.02 -0.015 -0.01 -0.005 0 0.005 0.01 0.015 0.02 ∆∆∆∆ y Readings Damage 400 600 800 1000 1200 1400 1600 1800 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 x 10-3 a m p li tu d e o f th e l o w e s t fr e q u e n c y Readings Damage +6σσσσ -6σσσσ
7
The next step is to increase the complexity in the measurement sets by including traffic loads (L2). The introduction of traffic loading has a great influence on the performance of SVR. ∆y values for measurements that comprise traffic loading are significantly larger than for those that were taken in the absence of traffic. Figure 6 shows the ∆y values for sensor S-3 under scenario TEMP1L2D1. The plot of ∆y for this scenario is much different to that for scenario TEMP1L1D1 (Figure 5). However, signal interpretation using MFFT reveals the anomaly.
Figure 6: Time series of the differences (∆y) between measured and predicted strain values at S-3 for scenario TEMP1L2D1 (left); Results from MFFT of the ∆y time series (right).
SVR has also been successfully tested for datasets of the other two temperature distributions previously mentioned in Section 2 and these results are summarized in the next sub-section. To ensure that these results would also be valid for more general temperature distributions that are possible in real bridges, a more complex temperature distribution – TEMP4, is created. TEMP4 is a combination of TEMP2 and TEMP3 and therefore includes two readings per day. The assumption is that TEMP2 and TEMP3 may represent temperature distributions during two different times during the day such as at dusk and at dawn. ∆y values for scenario TEMP4L3D1 are shown in Figure 7. As is to be expected, the plot shows that traffic loads resulting from light and heavy vehicles lead to significant differences between the predicted and measured strains.
Figure 7: ∆y for scenario TEMP4L3D1 (readings from S-3). The deviation from expected results increases depending on the intensity of the applied load.
0 500 1000 1500 -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05 ∆∆∆∆ y Readings Damage 400 600 800 1000 1200 1400 1600 1800 -3 -2 -1 0 1 2 3 4 5 6 7 8x 10 -3 a m p li tu d e o f th e l o w e s t fr e q u e n c y Readings Damage +6σσσσ -6σσσσ 0 500 1000 1500 2000 2500 3000 3500 -0.05 0 0.05 0.1 0.15 0.2 ∆∆∆∆ y Readings Damage
8
The large deviations in the SVR predictions due to traffic do not, however, significantly affect the performance of the SVR model. The prediction errors (∆y) from a SVR for scenario TEMP4L3D1 are shown in Figure 7. This time series is subsequently analyzed using MFFT. The amplitude of the lowest frequency from MFFT for this scenario is plotted in Figure 8. We could also use MPCA to analyze the time series of ∆y values. MPCA, however, uses the correlation between multiple time series to determine principal components. Therefore, it requires ∆y values for multiple strain sensors on the bridge. The component values of the principal eigenvector that correspond to sensor S-2 are plotted in Figure 8 for the same scenario TEMP4L3D1.
Figure 8: Design scenario TEMP4L3D1: ∆y interpretation using MPCA S-2 (left) and MFFT S-3 (right).
4.1 Discussion
The previous section presented notable results from this study for only a few scenarios. The research, however, investigated the SVR approach for a large set of scenarios. It also compared the performance of SVR with MPCA of the measurement time histories, which was previously proposed by Posenato et al. (2010). As expected, time to detect damage varies from scenario to scenario as well as the algorithm that is employed to analyze the time series of prediction error ∆y. A summary of results for a wide range of scenarios using load case L1 is presented in Table 1.
Table 1: Time to detection for load case L1.
TEMP1 TEMP2 TEMP4
L1 D1 L1 D2 L1 D3 L1 D1 L1 D2 L1 D3 L1 D1 L1 D2 L1 D3
Algorithm Time to detection (days)
ν-SVR
MPCA 19 40 145 20 41 195 22 100 173
MFFT 52 194 213 47 192 193 54 172 197
MPCA x x x x x x 248 250* 380*
* Weak evidence of an anomalous behaviour.
1000 1500 2000 2500 3000 3500 -0.05 -0.045 -0.04 -0.035 E ig e n v e c to r Readings Damage +6σσσσ -6σσσσ 1000 1500 2000 2500 3000 3500 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3x 10 -3 a m p li tu d e o f th e l o w e s t fr e q u e n c y Readings Damage +6σσσσ -6σσσσ
9
5. Conclusions
Conclusions from this study are as follows:
• SVR could form the basis of a methodology for detecting anomalies from distributed response (strain) and temperature measurements.
• Results show that SVR models could be more accurate and detect damage faster than interpretation of the measurement time histories using MPCA. The approach is also capable of detecting damage when measurements include the effects of both vehicular traffic and thermal variations.
• The prediction error, which is the difference between strains predicted by a SVR model and measurements, could be a reliable indicator of damage. After damage, this signal has a significant component that corresponds to the frequency of the seasonal temperature variations.
• MPCA and MFFT could be used to automatically detect damage from the time series of prediction errors.
Future research will involve testing the developed methods for measurements from full-scale structures. Work is underway on extending these approaches to find the location of damage. Further investigation is required on the sensitivity of the SVR-based approach for anomaly detection to tuning parameters such as ν. An appropriate outlier removal technique could also be used to cleanse the SVR predictions off measurements that incorporate effects of vehicular traffic. In the longer term, research could combine the developed methods with strategies that identify traffic loads on the structure.
References
Ansys (2011). ® Academic Research, Release 13.0, Help System, Mechanical APDL, Element Reference.
ANSYS, Inc.
Brownjohn, J. (2007). Structural health monitoring of civil infrastructure. Philos Transact A Math Phys Eng Sci.
, 365, 589–622.
Chandola, V., Banerjee, A. & Kumar, V. (2009). Anomaly Detection : A Survey. A modifed version of this
technical report will appear in ACM Computing Surveys, September 2009, 1-72.
Chang, C.-C. & Lin, C.-J. (2002). Training v-Support Vector Regression: Theory and Algorithms. Neural
Computation, 14, 1959-1977.
Chang, C.-C. & Lin, C.-J. (2011). LIBSVM : a library for support vector machines. ACM Transactions on
Intelligent Systems and Technology, 2, 27:1--27:27.
Cole, G. (2008). The role of assessments in highway bridge management. Proceedings of the ICE - Bridge
Engineering, 161, 133-139.
Heller, K. A., Svore, K. M., Keromytis, A. D. & Stolfo, S. J. (2003). One Class Support Vector Machines for Detecting Anomalous Windows Registry Accesses. In Proc. of the workshop on Data Mining for Computer
Security.
Klein Tank, A. M. G., Wijngaard, J. B., Können, G. P., Böhm, R., Demarée, G., Gocheva, A., Mileta, M., Pashiardis, S., Hejkrlik, L., Kern-Hansen, C., Heino, R., Bessemoulin, P., Müller-Westermeier, G., Tzanakou, M., Szalai, S., Pálsdóttir, T., Fitzgerald, D., Rubin, S., Capaldo, M., Maugeri, M., Leitass, A., Bukantis, A., Aberfeld, R., Van Engelen, A. F. V., Forland, E., Mietus, M., Coelho, F., Mares, C., Razuvaev, V., Nieplova, E., Cegnar, T., López, J. A., Dahlström, B., Moberg, A., Kirchhofer, W., Ceylan, A., Pachaliuk, O., Alexander, L. V. & Petrovic, P. (2002). Daily dataset of 20th-century surface air temperature and precipitation series for the European Climate Assessment. International Journal of Climatology, 22, 1441-1453.
Laory, I., Trinh, T. N. & Smith, I. F. C. (2011). Evaluating two model-free data interpretation methods for measurements that are influenced by temperature. Advanced Engineering Informatics, 25, 495-506.
10
Mirambell, E. & Aguado, A. (1990). Temperature and Stress Distributions in Concrete Box Girder Bridges
Journal of Structural Engineering 116, 2388-2409.
Moorty, S. & Roeder, C. W. (1992). Temperature‐Dependent Bridge Movements Journal of Structural
Engineering, 118, 1090.
Ni, Y. Q., Hu, X. G., Fan, K. Q. & And Ko, J. M. (2005). Correlating modal properties with temperature using long-term monitoring data and support vector machine technique. Engineering Structures, 27, P. 1762-1773. Ni, Y. Q., Zhou, H. F., Chan, K. C. & Ko, J. M. (2008). Modal Flexibility Analysis of Cable-Stayed Ting Kau Bridge for Damage Identification. Computer-Aided Civil and Infrastructure Engineering, 23, 223-236.
Noori, M. N., Cao, Y., Hou, Z. & Sharma, S. (2010). Application of Support Vector Machine for Reliability Assessment and Sturctural Health Monitoring. International Journal of Engineering Under Uncertainty:
Hazards, Assessment and Mitigation, 2, 89-98.
Posenato, D., Kripakaran, P., Inaudi, D. And Smith, I.F.C. (2010). Methodologies for model-free data interpretation of civil engineering structures. Computers and Structures, 88, 467-482.
Saitta, S., Kripakaran, P., Raphael, B. & Smith, I. F. C. (2010). Feature Selection Using Stochastic Search: An Application to System Identification Journal of Computing in Civil Engineering 24, 3-10.
Schölkopf, B., Platt, J., Shawe-Taylor, J., Smola, A. J. & Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, 13, 1443-1471.
Shengchao, S., Na, D. & Gang;, H. Y. An SVR-Based Online Fault Detection Method 2011 Third International Conference on Measuring Technology and Mechatronics Automation, 2011 Shangshai 447 - 450
Sherlock, B. G. & Monro, D. M. (1992). Moving discrete Fourier transform. Radar and Signal Processing, IEE
Proceedings F 139.
Yuen, K.-V. & Kuok, S.-C. (2010). Ambient interference in long-term monitoring of buildings. Engineering
Structures, 32, 2379–2386.
View publication stats View publication stats
