Project Description

Anomaly diagnosis (i.e., anomaly detection, identification, and quantification) in computer networks is a challenging and important problem. Anomalies can arise for many reasons: unusual end-user demands; network misuse and abuse; equipment failure; and operational misconfiguration. Our thesis is that the entire range of such problems can only accurately be diagnosed via a whole-network approach: i.e., by examining traffic throughout a network simultaneously. This is in contrast to previous work which has almost exclusively dealt only with traffic measured at single point in a network.

While the whole-network approach is powerful, it is challenging because of the need to extract meaning from noisy, high-dimensional data. To that end the project has already made three contributions. In [1] we characterized the set of traffic observable across a network and showed that, even in a network with hundreds of flows, most significant traffic variation can be capture in a small number of dimensions (less than 10). In [2] we showed how to exploit the low effective dimensionality of network traffic for anomaly detection using the subspace approach. Finally, in [3] we show that the subspace approach is remarkably powerful at detecting the entire range of network anomalies when applied to traffic flow data.