...
A step towards this is the low level analysis of performance data for anomaly detection. Applied to say, throughput between two sites, measurements can be fitted against various distributions and outliers flagged as being potential performance alerts. We have prepared a paper that was is being submitted to Infocom to be published IMC 09, with application towards generic data as proof of concept. The first draft of the technical report is available online 5. N.b the paper submitted to IMC proposes an approach with significant revisions to the methods suggested in the first draft - which was compiled in July '08.
In addition, the collection and cross correlation of various network metrics (that perfSONAR provides) in order to identify a performance 'culprit' can provide an extensive method of wading through the vast amounts of data. It will also provide a means by which problem tickets can provide more useful information than simply that something is wrong 6.We first experimented with Multi-route Event Detection using PCS 6, however the results were disappointing. We then looked at mining the events detected to look for correlations and reduce false positives and negatives and provide improved diagnostics 7. Though it had some successes, the work should be classified as only the preliminay investigation of the overall problem.
1 The PingER Project: 1 Active Internet Performance Monitoring for the HENP Community, IEEE Communications Magazine on Network Traffic Measurements and Experiments.
2 perfsonar modules Internet2 Software
3 live cd Performance Node: Live CD
4 Google Maps Mashup http://psvis0.internet2.edu:8008/
5 Technical Report: A Decision-Theoretic Approach to Detect Anomalies in Internet Paths, by Fida HusseinHussain, Umar Kalim, Noman Latif, Syed Ali Kayam, submitted to Infocom, 2008.
6 adnanKhayam. A revised approach has been proposed and is being submitted to IMC 09.
6 Multi-route Event Detection Using PCA, Adnan Iqbal, unpublished, available at http://confluence.slac.stanford.edu/display/IEPM/Multi-route+Event+Detection+Using+PCA.
7 Event Diagnosis Adnan Iqbal, accepted for ICIMP 2007 (San Jose), but withdrawn, also see http://confluence.slac.stanford.edu/display/IEPM/Event+Diagnosis and the presentations referenced therein.