A semantics-based approach to malware detection

Mila Dalla Preda, Mihai Christodorescu, Somesh Jha, Saumya Debray

Research output: Contribution to journalArticle

65 Scopus citations

Abstract

Malware detection is a crucial aspect of software security. Current malware detectors work by checking for signatures, which attempt to capture the syntactic characteristics of the machine-level byte sequence of the malware. This reliance on a syntactic approach makes current detectors vulnerable to code obfuscations, increasingly used by malware writers, that alter the syntactic properties of the malware byte sequence without significantly affecting their execution behavior. This paper takes the position that the key to malware identification lies in their semantics. It proposes a semantics-based framework for reasoning about malware detectors and proving properties such as soundness and completeness of these detectors. Our approach uses a trace semantics to characterize the behavior of malware as well as that of the program being checked for infection, and uses abstract interpretation to "hide" irrelevant aspects of these behaviors. As a concrete application of our approach, we show that (1) standard signature matching detection schemes are generally sound but not complete, (2) the semantics-aware malware detector proposed by Christodorescu et al. is complete with respect to a number of common obfuscations used by malware writers and (3) the malware detection scheme proposed by Kinder et al. and based on standard model-checking techniques is sound in general and complete on some, but not all, obfuscations handled by the semantics-aware malware detector.

Original languageEnglish (US)
Article number25
JournalACM Transactions on Programming Languages and Systems
Volume30
Issue number5
DOIs
StatePublished - Aug 1 2008

Keywords

  • Abstract interpretation
  • Malware detection
  • Obfuscation
  • Trace semantics

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'A semantics-based approach to malware detection'. Together they form a unique fingerprint.

  • Cite this