US 7,555,777 B2
Preventing attacks in a data processing system
Morton D. Swimmer, Waedenswil (Switzerland); Andreas Wespi, Niederhasli (Switzerland); and Diego M. Zamboni, Waedenswil (Switzerland)
Assigned to International Business Machines Corporation, Armonk, N.Y. (US)
Filed on Jan. 13, 2004, as Appl. No. 10/756,744.
Prior Publication US 2004/0255163 A1, Dec. 16, 2004
Int. Cl. G06F 11/30 (2006.01)
U.S. Cl. 726—23 7 Claims
OG exemplary drawing
 
1. A method for preventing attacks in a monitored data processing system comprising the steps of:
detecting an intrusion into the data processing system by monitoring system calls from a daemon executed in a memory of the monitored data processing system;
upon detection of an intrusion, identifying a malicious code string related to the detected intrusion by matching the system calls with one or more of established patterns and rules contained in a pattern matcher and representing a model of normal behaviour, wherein the matching of the system calls comprises establishing a non-deterministic automaton based on an analysis of executable code of the daemon;
extracting the malicious code string by the steps of:
intercepting the system call via a subprogram of the sensor for observing the interaction of the daemon and the operating system;
inspecting a stack upon detection of an intrusion to retrieve an address leading to the malicious code string;
locating, as a first element on the stack, a return address of a system call entry code from which the subprogram departed; and
retrieving a return address of the malicious code string pointing to a memory location in the range in which the daemon is executed from a second element on the stack positioned at or near the location of the return address of the system call entry code to facilitate finding and extracting of the malicious code string;
scanning the memory range owned by the executed daemon starting from the return address in opposite directions until on one side a first region with a plurality of similar addresses and on the other side a second region with a plurality of similar instructions that do not alter the sequential control flow is identified; and
extracting the malicious code string from between the first and second regions;
forwarding the malicious code string to an intrusion limitation subsystem to reduce further intrusions based on the malicious code string.