University of Houston

A common technique hackers use to avoid being detected is to route their network connections through a chain of stepping-stone hosts. There is no valid reason to use a long connection chain for remote login such as SSH connection. In this dissertation, we focus on protecting hosts from being attacked via stepping-stone connection chains. Our objective is to detect intruders at a stepping-stone host in the middle of the connection chain and at the target host at the end of the chain.

Along with the developing of correlation based stepping-stone detection algorithms, hackers also developed new techniques to evade being detected. Hackers can add chaff packets or jitter the original packets to decrease the detection rate of these correlation algorithms. Our jittering detection algorithm utilizes statistical distributions to fit the inter-arrival time gaps of traffic flows, extracting features from fitting, and separates jittered ones from normal ones by using SVM. We further propose a hybrid stepping-stone detection algorithm to employ both correlation and jitter detection algorithms to detect intrusions.

It is always important for a host to protect itself from being a victim. To detect long connection chain intrusions at the target host, we propose two detection algorithms: a nearest neighbor based algorithm and an anomaly detection based algorithm. The first algorithm uses an approximated upstream round-trip time to separate a long connection chain from short ones. Besides, based on the idea of anomaly behavior detection, a novel method to identify long connection chains from short chains using a pre-defined short chain profile has been proposed.

With the algorithms proposed in this dissertation, we can detect stepping-stones in the middle of the chain in a robuster way, and we can further and more effectively protect victim hosts from stepping-stone intrusions at the end of the chain.