Analysis and mitigation of evasive behavior in malicious software
With thousands of new viruses and other malware threats surfacing every day, dynamic analysis techniques play a fundamental role in the automatic characterization and detection of malicious behaviors that undermine the security of computing systems. Modern malware has started however to adopt reconnaissance techniques that fingerprint the execution environment, looking for possible artifacts that could reveal the presence of a monitoring system in lieu of a plausible victim, and consequently hiding its true colors to elude detection and analysis.
In the arms race between malware writers and analysis systems, researchers and security firms have invested in stealth execution environments that use hardware virtualization and cloaked software configurations to execute and analyze malware dynamically. And yet, often new threats emerge that defeat such automatic analyses, requiring them to be manually dissected by expert analysts.
We propose a methodology for hardening automatic dynamic analysis to make it more robust against evasive malware. Our approach is unprecedented: using dynamic binary instrumentation, we look closely at each fingerprinting attempt made by a malicious program as it executes, and dynamically choose the best possible answer to provide, rewriting program behaviors that are instead fixed in extant solutions. Further novelty lies in the pivotal role of the human element in our methodology: we wish to boost the productivity of analysts that intervene to defeat unprecedented evasions, and we provide means to incorporate their findings in the automatic analysis system as part of a human-assisted feedback loop mechanism.
Preliminary experimental results suggest that our approach is well-equipped to deal with the more advanced evasion techniques from highly evasive malware that even sophisticated commercial solutions have recently struggled to deal with.
The project will be carried out as part of a joint effort with King's College London.