Overview
In this project we study and investigate network anomaly detection algorithms [1] [2] [3] for Internet Paths. We also develop a Decision Theoretic Approach based on our observations about the characteristics of the performance measurements statistics obtained from the IEPM-BW project.
To study and compare the algorithms we use the data sets collected by IEPM-BW spanning approximately 2 years (i.e. 2006 - 2008). The Internet paths observed were the links between Stanford Linear Accelerator Center (SLAC) and the following sites:
- San Diego Supercomputing Center (SDSC) USA,
- Oak Ridge National Laboratory (ORNL) USA,
- European Organization for Nuclear Research (CERN) Geneva, Switzerland,
- Forschungszentrum Karlsruhe (FZK) Germany,
- Deutsches Elektronen- Synchrotron (DESY) Germany and
- University of Toronto (UTORONTO) Canada.
Data Sets
The data sets used in the study may be downloaded from the links listed below. These data sets were collected by the IEPM-BW project and the latest performance statistics may be accessed from here.
|
Raw data |
Labeled data |
|||||
---|---|---|---|---|---|---|---|
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="866d8f35-3437-4765-823a-2b91cff8754b"><ac:plain-text-body><![CDATA[ |
SDSC |
[[csv |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/SDSC-pathchirp.csv]], [[xls |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/SDSC-pathchirp.xls]] |
[[txt |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/UTORONTO-pathchirp-labeled-events.txt]] |
]]></ac:plain-text-body></ac:structured-macro> |
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="d05d3d2d-a616-4390-9383-bb2f40bcdb9d"><ac:plain-text-body><![CDATA[ |
CERN |
[[csv |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/CERN-pathchirp.csv]], [[xls |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/CERN-pathchirp.xls]] |
[[txt |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/CERN-pathchirp-labeled-events.txt]] |
]]></ac:plain-text-body></ac:structured-macro> |
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="a6359fcd-9fee-458c-a10c-f3b87ae6ad5c"><ac:plain-text-body><![CDATA[ |
FZK |
[[csv |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/FZK-pathchirp.csv]], [[xls |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/FZK-pathchirp.xls]] |
[[txt |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/FZK-pathchirp-labeled-events.txt]] |
]]></ac:plain-text-body></ac:structured-macro> |
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="296de8d4-aabd-4fc4-b61f-5b126bcc3fbf"><ac:plain-text-body><![CDATA[ |
DESY |
[[csv |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/DESY-pathchirp.csv]], [[xls |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/DESY-pathchirp.xls]] |
[[txt |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/DESY-pathchirp-labeled-events.txt]] |
]]></ac:plain-text-body></ac:structured-macro> |
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="3a8ddfe9-de10-4752-b540-c6ebb4083b60"><ac:plain-text-body><![CDATA[ |
UTORONTO |
[[csv |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/UTORONTO-pathchirp.csv]], [[xls |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/UTORONTO-pathchirp.xls]] |
[[txt |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/UTORONTO-pathchirp-labeled-events.txt]] |
]]></ac:plain-text-body></ac:structured-macro> |
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="b4cccc64-b323-4866-b40f-78dbb54266c0"><ac:plain-text-body><![CDATA[ |
ORNL |
[[csv |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/ORNL-pathchirp.csv]], [[xls |
http://www.slac.stanford.edu/~kalim/event-detection/published-data/ORNL-pathchirp.xls]] |
[txt] |
]]></ac:plain-text-body></ac:structured-macro> |
Download the complete data archive [zip 11 MB].
Labeling Algorithm
The labeling algorithm is as under:
Implementations and Parameter Tuning
The source code of the implementations and the tuning of parameters is discussed below.
References
- C. Logg, L. Cottrell, and J. Navratil. Experiences in traceroute and available bandwidth change analysis. In NetT '04: Proceedings of the ACM SIGCOMM workshop on Network troubleshooting, pages 247-252. ACM, 2004.
- A. Soule, K. Salamatian, and N. Taft. Combining filtering and statistical methods for anomaly detection. In Internet Measurement Conference (IMC 2005), pages 331-344. USENIX, 2005.
- H. Hajji. Statistical analysis of network traffic for adaptive faults detection. In IEEE Transactions on Neural Networks, pages 1053-1063, 2005.