Two studies of operating system security by the School of Computer Science at the Georgia Institute of Technology reveal widespread kernel problems: one study detects a vulnerability believed to affect all devices using “Intel TSX” (Intel Transactional Synchronization Extension) to aid memory processing, while the other study introduces a tool to prevent previously unknown information leaks from the Linux operating system kernel.

Both studies – which broadly impact not only Intel TSX, but also Linux and Android systems – were presented this week by students and faculty from the School of Computer Science at the 23rd ACM Conference on Computer and Communications Security (CCS) in Vienna, Austria.

The operating system kernel is the de facto trusted computing base for most computer systems. Attacks on the kernel can corrupt memory, leading to more crashes as the computer attempts to access recent data and cannot do so. Insecure kernels also can passively or actively leak sensitive information that a user would prefer to keep private.

In “Breaking KASLR with Intel TSX,” Assistant Professor Taesoo Kim – working with Ph.D. students Yeongjin Jang and Sangho Lee – attempted to de-randomize one of the latest solutions from Intel designed to protect its devices. In doing so, they turned an exploit into a precise timing channel attack called “DrK” that can break all operating systems in under one second with no visible footprint.

“The root cause of this security loophole, is that it aborts a transaction without notifying the underlying kernel even when the transaction fails due to a critical error,” stated the authors. DrK uses that opening to determine the mapping status (i.e., mapped versus unmapped) and execution status (i.e., executable versus non-executable) of the privileged, kernel address space. Their findings were reported to the United States Computer Emergency Readiness Team at the Department of Homeland Security, Intel and Microsoft. Intel since has issued an errata advisory about its 6th generation processor family.

In “UniSan: Proactive Kernel Memory Initialization to Eliminate Data Leakages,” Ph.D. student Kangjie Lu studied why most kernel information leaks are caused by uninitialized data reads. He found that, even when developers correctly write their code, compilers can introduce uninitialized data, which have caused many information leaks. Unfortunately, existing techniques like memory safety enforcements and dynamic access tracking tools are not adequate or efficient enough to mitigate this threat.  Lu and fellow authors – Chengyu Song (Ph.D. CS ’16), Kim and Professor Wenke Lee – created UniSan, a compiler-based approach to eliminate information leaks caused by uninitialized read in the OS kernel.

UniSan achieves its goal using byte-level, flow-sensitive, context-sensitive, and field-sensitive initialization analysis to check whether an allocation has been fully initialized when it leaves kernel space. If not, UniSan automatically instructs the kernel to initialize it. When UniSan was applied to the latest Linux kernel (x86_64) and Android kernel (AArch64), it successfully prevented 43 known vulnerabilities and found 19 new vulnerabilities that have since been confirmed by Linux and Google. Their findings also have triggered extensive discussions in the Linux community and many patches in the Linux kernel.

Finding vulnerabilities – whether through active attacks, such as DrK, or through new prevention tools like UniSan – helps to make systems and devices more secure, explains Wenke Lee, co-director of the Institute for Information Security & Privacy.

“Anytime you find a vulnerability in hardware or software, you want to take it seriously and report it,” he says. “These platforms are used by so many, and security researchers have a responsibility to make sure they remain secure. Attackers are going to learn this on their own and share nothing with the rest of us, so it is always better that vulnerabilities are discovered by security researchers, reported so that vendors can fix them, and shared so others can learn from it.”

Both Jang and Lu say there is great satisfaction in knowing their work can make technology products more secure.

“Many think that the bad people will benefit, but users at least know,” Jang says.