University of Houston

Intrusion detection plays a surveillant role by identifying attacks and protecting information systems from unauthorized access, misuse or disruption. This dissertation investigates several topics in stepping-stone and masquerader intrusion detections. Network intruders usually indirectly launch attacks by constructing a long connection via intermediary hosts, called stepping-stones, to evade detection. Two detection approaches, size-fluctuation and random walk with transformation, are presented to identify whether a host has been used as a stepping-stones. We focus on some sophisticated intruders who add extra superfluous packets to disturb the detection.

Once a target user’s machine is invaded by intruders, the interlopers may impersonate legitimate user’s account to gain access to unauthorized computer systems. This is called a masquerade problem. Two anomaly detection models are developed to differentiate a legitimate user from a masquerader. The first one is the high frequency command approach that profiles the behavior of a user on a computer system according to the most frequently used commands. The second one is the command prediction with association rule mining approach that builds user’s behavior pattern in order to predict a masquerader’s next command. We further investigate the relationship between the user behavior in terms of operating system commands and the success rate of detection.