We know that software flaws can create vulnerabilities within a system. But a system’s design often impacts whether a software flaw can be exploited. For example, subtle differences between a system’s ideal design and its implementation can lead to different emergent run-time behaviors. These emergent behaviors can act like a “programmable weird machine” – making a system susceptible to routine exploitation.
One of our latest projects at Galois, Discovering and Explaining Threats of Emergent Computation to enable Transformations (DETECT), aims to help programmers discover vulnerabilities within a system’s design. As part of DARPA’s HARDEN program, DETECT builds upon SEEC, a Galois project that was part of DARPA’s AIMEE program. SEEC used ideas from the secure compilation and program synthesis fields to detect and reason about weird machines without pre-existing knowledge of potential flaws in a system.
DETECT expands on SEEC in several ways. DETECT will compare the behavior of computations at different levels of abstraction in models created by other performers within the HARDEN program. DETECT will also improve SEEC’s scalability by replacing program synthesis techniques with techniques from state-of-the-art differential fuzzing tools. (Fuzzing is a technique that systematically generates inputs for a program with the goal of identifying invalid or unexpected behaviors.) DETECT will also help identify specific design decisions that lead to emergent computation. This will allow system designers to fix these problems earlier in the design process.
Once developers have refactored their design or implementation to fix a source of emergent computation, DETECT will re-execute generated test cases to validate that the changes have disrupted the emergent computation. If the changes are successful, the developer can return to a previous step of DETECT workflow to find and repair other sources of emergent computation.
You can dig into the details about DETECT on its project page.
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) and Naval Information Warfare Center Pacific (NIWC Pacific) under N66001-22-C-4027. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of DARPA and NIWC Pacific.
Distribution Statement “A” (Approved for Public Release, Distribution Unlimited). If you have any questions, please contact the Public Release Center.