Title: Security and bug-finding in foreign function interfaces and multi-language programs

Date: Friday, April 18, 2025

Time: 1:00 PM – 3:00 PM EST

Location: (hybrid) CODA 1006 West End, and Zoom (https://gatech.zoom.us/j/98789993263)

Ammar Askar

School of Computer Science & School of Cybersecurity and Privacy

College of Computing

Georgia Institute of Technology

Committee:

Dr. Taesoo Kim (advisor), School of Cybersecurity and Privacy, Georgia Institute of Technology

Dr. Brendan D. Saltaformaggio, School of Cybersecurity and Privacy, Georgia Institute of Technology

Dr. Qirun Zhang - School of Computer Science, Georgia Institute of Technology

Dr. Alessandro Orso - School of Computer Science, Georgia Institute of Technology

Dr. Sangho Lee - Redmond Security Research Group, Microsoft Research

Abstract:

Modern programs often require the use of multiple programming languages. High level languages like Java and Python allow rapid prototyping and fast development speeds without having to worry about low level details such as memory management. However, due to performance constraints, and to interface with many foundational libraries, it is sometimes necessary to write code in low level languages such as C.

Due to the differences in the semantics and security considerations in different languages, there is a likelihood of security issues being introduced when programmers context-switch between writing in multiple languages. For example, a programmer who is used to array accesses being bounds-checked by the language may end up introducing a spatial memory-safety issue with an out of bounds access. One who is unfamiliar with the nuances of manual memory management may introduce a use-after-free vulnerability.

This thesis presents how the use of synthesized fuzzing harnesses can explore program and data flow in one language that triggers bugs in the foreign language. We also propose a novel new method of applying concolic execution to a variety of programming languages. Through this technique, the process of implementing concolic execution only requires simple debugging primitives and uses LLMs to ease the implementation burden. This technique can be applied to multi-language programs to find bugs between them.