A Diagnostic Approach for Persistent Threat Detection (ADAPT)

DARPA’s Transparent Computing program aims to make currently opaque computing systems transparent by providing high-fidelity visibility into component interactions during system operation across all layers of software abstraction, while imposing minimal performance overhead. To accomplish this goal, ADAPT, a collaboration led by Galois that also includes the University of Edinburgh, Synaptiq, and Oregon State University, aims to:

• track causality across the enterprise and over extended periods of time,
• identify subtle causal chains that represent malicious behavior,
• localize the code at the roots of such behavior,
• trace the effects of other malicious actions descended from those roots, and
• make recommendations on how to mitigate those effects.

ADAPT uses provenance to help in understanding influence of one computational process on another. For example, one process may create another, or write a file that is later read by another. Either way, the former process becomes part of the provenance of the latter, able to influence its behavior in direct or subtle ways. Tracing this provenance through multiple generations of influence is a way of looking into the history of a computation, identifying the many ancestors that might influence it – the “family tree” of a computation. In a complex system, provenance analysis can tease apart the interwoven interconnections of these family trees, much as the human eye can distinguish pictures hidden within other pictures.

If one or more of these computational family trees stands out among the rest as unusual in some way, it may be an APT. ADAPT uses statistical anomaly detection to detect patterns that stand out in a crowd. Statistical analysis of file access can for example identify processes that “scan” many files relative to other similar processes. Combining the results of many anomaly detectors, and associating their results with computational family trees revealed by provenance analysis, can distinguish some family trees as having higher “anomaly scores” than others. However, a computational family tree with notable anomaly scores may still not be an APT.

When computational family trees that have high anomaly scores also match broad patterns of APT behavior, they should come to the front of the list for attention by network defense personnel. ADAPT uses artificial intelligence approaches to diagnose different family tree behaviors, singling out those that match the broad strokes of APTs. ADAPT then prioritizes those family trees by a combination of anomaly scores and strength of match against those broad patterns, and presents the result for action by human or machine network defenders.

By doing so, ADAPT aims to enable stakeholders to understand and manage the activities going on in their networks. ADAPT leverages both current and novel forms of local causality to construct graphs of global, long-term causality in system information flow. ADAPT classifies segments of such graphs by the activities they represent, and reasons over these activities, prioritizing candidate activities for investigation. ADAPT’s diagnostic engine investigates these candidates looking for patterns that may represent the presence of APTs. ADAPT leverages models of APT and normal business behavior to diagnose such threats. ADAPT is not constrained by availability of human analysts, but can benefit by human-on-the-loop assistance.