Detecting a botnet in a network

Abstract

We formalize the problem of detecting the presence of a botnet in a network as a hypothesis testing problem where we observe a single instance of a graph. The null hypothesis, corresponding to the absence of a botnet, is modeled as a random geometric graph where every vertex is assigned a location on a $d$-dimensional torus and two vertices are connected when their distance is smaller than a certain threshold. The alternative hypothesis is similar, except that there is a small number of vertices, called the botnet, that ignore this geometric structure and simply connect randomly to every other vertex with a prescribed probability.

We present two tests that are able to detect the presence of such a botnet. The first test is based on the idea that botnet vertices tend to form large isolated stars that are not present under the null hypothesis. The second test uses the average graph distance, which becomes significantly shorter under the alternative hypothesis. We show that both these tests are asymptotically optimal. However, numerical simulations show that the isolated star test performs significantly better than the average distance test on networks of moderate size. Finally, we construct a robust scheme based on the isolated star test that is also able to identify the vertices in the botnet.