Using Sub-optimal Kalman Filtering for Anomaly Detection in Networks

Joseph Ndong


Possibility theory can be used as a suitable frameworkto build a normal behavioral model for an anomaly detector.Based on linear and/or nonlinear systems, sub-optimal filteringapproaches based on the Extended Kalman Filter and the UnscentedKalman Filter are calibrated for entropy reduction andcould be a good basis to find a suitable model to build a decisionvariable where, a decision process can be applied to identifyanomalous events. Sophisticated fuzzy clustering algorithms canbe used to find a set of clusters built on the decision variable,where anomalies might happen inside a few of them. To achievean efficient detection step, a robust decision scheme is built, bymeans of possibility distributions, to separate the clusters intonormal and abnormal spaces. We had studied the false alarmrate vs. detection rate trade-off by means of ROC (ReceiverOperating Characteristic) curves to show the results. We validatethe approach over different realistic network traffic.


Extended Kalman Filter, Unscented Kalman Filter, Fuzzy Clustering, Anomaly Detection, Possibility theory.


Dubois, D., Prade, H. and Sandri, S.: On possibility/probability transformations. In Proceedings of the Fourth Int. Fuzzy Systems Association World Congress (IFSA91), Brussels, Belgium, pages 50-53, (1991).

Maybeck, P. Stochastic Models, Estimation and Control, Volume 2. Academic Press. 1982. Using MATLAB. Wiley Interscience. 2001.

Bar-Shalom, Y., Li, X.-R., and Kirubarajan, T. Estimation with Applications to Tracking and Navigation. Wiley Interscience. 2001.

Masson, M., H. and Denoeux, T.: Inferring a possibility distribution from empirical data. Fuzzy Sets and Systems 157(3): pp. 319-340, 2006.

Zadeh, L.,A. Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets and Systems, 1: pp. 3-28, 1978.

Bezdek, J. C. Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, 1981.

Bensaid, A. M., Hall, L.O, Bezdek, J.C., Clarke, L.P., Silbiger, M.L., Arrington, J. A and Murtagh, R.F Validity-guided (Re)Clustering with applications to image segmentation. IEEE Transactions on Fuzzy Systems, 4:112- 123, 1996.

Lakhina, A., Crovella, M. and Diot, C.: Characterization of networkwide traffic anomalies. In Proceedings of the ACM/SIGCOMM Internet Measurement Conference. pp. 201-206.(2004)

Lakhina, A., Crovella, M.,Diot, C.: Diagnosing Network-Wide Traffic Anomalies. In ACM SIGCOMM (2004).

Xie X.,L. and Beni, G. A. Validity measure for fuzzy clustering. IEEE Trans. PAMI, 3(8):841-846, 1991.

Wan, E. and Van Der Merwe, R. The Unscented Kalman Filter. Wiley Publishing, 2001.

Babuska, R., Van der Veen, P. J. and Kaymak, U: Improved covariance estimation for Gustafson-Kessel clustering. IEEE International Conference on Fuzzy Systems, pages 1081-1085, 2002.

Goodman, L. A.: On simultaneous confidence intervals for multinomial proportions. Technometrics, 7(2): pp. 247-254, 1965.

Ndong, J., Salamatian, K., :A Robust Anomaly Detection Technique Using Combined Statistical Methods. CNSR 2011, IEEE Xplore 978-1- 4577-0040 8, pp: 101-108. (May 2011).

Ndong, J., Salamatian, K.,: Signal Processing-based Anomaly Detection Techniques: A Comparative Analysis. INTERNET 2011, The Third International Conference on Evolving Internet. ISBN: 978-1-61208-141- 0.

Ndong, J.: Anomaly Detection: A Technique Using Kalman Filtering and Principal Component Analysis. ATAI NTC 2012 GSTF 2012.

Full Text: PDF


  • There are currently no refbacks.