Title: Efficient Monitoring and Attribution of Malicious Behaviors

Abhinav Srivastava
Georgia Tech Information Security Center
School of Computer Science
Georgia Institute of Technology


Committee:
Prof. Jonathon Giffin (Advisor, School of Computer Science, Georgia Institute of Technology)
Prof. Mustaque Ahamad (School of Computer Science, Georgia Institute of Technology)
Prof. Patrick Traynor (School of Computer Science, Georgia Institute of Technology)
Prof. Wenke Lee (School of Computer Science, Georgia Institute of Technology)


Thesis Summary:
Worldwide computer systems continue to execute software that exhibits malicious network and host behaviors. On networks, the visible effects of current attacks regularly manifest as suspicious traffic. On hosts, malware installs malicious kernel drivers, subverts the execution of benign processes (parasitic behaviors), and tampers with the existing host-based security utilities. The traditional host-based security software is unable to detect current generation malware. These security solutions are designed to detect and prevent application-level attacks. Current attacks regularly bypass existing protections by installing themselves in the kernel and invoking kernel functionality directly. They use kernel code illegitimately and modify kernel data illicitly. To counter these malware, it is required to monitor behaviors of kernel malware and protect kernel data from them.

Network-based detectors can effectively identify machines participating in the ongoing attacks by monitoring the traffic to and from the systems. However, they fail to determine the malicious processes associated with the suspicious traffic. Host-based detectors can identify malicious processes, but they are often disabled by knowledgeable attackers. The knowledge of identifying malicious processes attached to suspicious traffic creates the foundation for successful remediation.

My research focuses on attributing malicious network behaviors to host-level software and monitoring malicious behaviors occurring at user- and kernel-level. The proper attribution of malicious behaviors creates the foundation for subsequent surgical remediation of the malware infection. The ability to observe the execution of untrusted or malicious drivers improves the overall security of operating systems. In order to resist direct attacks from kernel-level malware, I take advantage of layers beneath OS code, such as a hypervisor or virtual machine monitor (VMM).

This dissertation proposal describes four unique contributions in host-based computer security. In the first contribution, I attributed malicious network behaviors to host-level processes associated with the malicious traffic. This successful attribution allowed me to create a tamper-resistant application-level firewall. Though the attribution identifies malicious processes, malware instances often exhibit parasitic behaviors in which they inject malicious code into benign processes to subvert their runtime behaviors. In my second contribution, I augmented the attribution software with a host-level monitor that detects parasitic behaviors occurring at user- and kernel-level. In my third contribution, I designed a system that monitors the execution of untrusted drivers. It isolates drivers in a separate address space, rewrites binary kernel and driver code at runtime, and generates new code on demand to reduce the monitoring overhead. Finally, in my last contribution, I am designing a system that prevents illegal modifications of critical kernel data from malicious drivers. Together, these contributions produce a unified research goal -- improving host-based security against user- and kernel-level malware