VM:EuroSys10 paper

From Trusted Cloud Group
Jump to: navigation, search



Spad: Software Protection through Anti-Debugging Based on Hardware Virtualization
候选名称:InvGuard(全称Invisible Guardian,隐形卫士);EiS(全称Eye in the Sky)


Debugging is a method that usually facilitates the dynamic analysis of run-time application for software development, but it could also be adopted as evil tools, especially with malicious attackers. Currently, although there exist various anti-debugging productivity tools, the anti-debugging problem still arise because the traditional software anti-debugging owns no ability to utilize higher privilege level than kernel space.

This paper proposes Spad (Software Protection through Anti-Debugging) to detect the behavior of debugger and disable their functionalities as well as make them imperceptible. Spad detects debugging behavior by intercepting debug instructions such as INT3 and similar operations. Since debuggers and malicious code could usually apply kernel space resource in parasitic operating system, our hypervisor based on hardware virtualization runs on root mode and plays an observer role on a more privileged position.

Experiment with several popular and typical debuggers shows that Spad can effectively disable the functionalities of debuggers. The vertical experiment also demonstrates that our design goes beyond most of the prevailing anti-debugging software. We further discuss related techniques and future work on anti-debugging strategy.

Keywords: anti-debugging, hardware virtualization, transparency, lightweight

1. Introduction

  • copyright protection的重要性
  • 黑客利用dynamic analysis破解正版软件(需要补充实例和reference引用)
    • example1
    • example2
    • ...
  • debugging是dynamic analysis的基本技术手段,所以我们要anti-debugging
  • 目前操作系统的限制,导致anti-debugging很难是完备的。(按原论文写法)
    • hardware architecture limit
    • bugs & debugging interfaces in OS
  • 目前已知的anti-debugging技术存在一些不足
    • 目前的anti-debugging技术基本可分为两类:
      • detect debugging registers
      • monitor human behaviors
    • 不足:
      • 每种anti-debugging都是针对某一特定的debugging软件或者技术,不完备。
      • 自身的隐藏性不好,容易被发现。
  • our task 简述,突出对比优势
  • Contributions:
    • 提出了一个XX、YY、ZZ的框架
    • 通过实验,证明了our task是具备实际应用价值的
    • (待补充)
  • following sections

In recent years, the requirement for software protection has gained highly attention in digital world. A great variety of copy-protection strategies of different forms have been developed to prevent stealth, decoding and reverse engineering. Major of these mechanisms provide a reasonable level of security against static-only analysis. Nevertheless, most of them avoid the issue of dynamic or hybrid static-dynamic attacks which are now commonly used by hackers.[20]

As mentioned in Gagnon’s article [1], current study on anti-reverse engineering and anti-dynamic analysis emphasizes on obfuscation, virtual machine detection, anti-disassembly, and tamper proofing, but little thought of anti-debugging. Debugging, as other program testing techniques, is widely employed as an essential tool in malicious code.

此处举例说明hackers如何利用debugging来进行dynamic attack,需要reference。
A detailed description of cracking password protection mechanism is presented in Kris Kaspersky’s book [23]. A hacker uses debugging method to set a breakpoint at password input function, look up for correct buffer and wait for the debugger window to show password up. Similar scenario may occur in other private and sensitive account data.

The risk of dynamic attacking on target software with debugging technique exists for the following reasons:

First, the hardware architecture is defective so that applications running on the privileged mode can access the whole system, while the code in the user mode can only own a little of resource [21]. That means once the malicious code or analyzing tools are running on the privileged mode, no higher mode can be used to restrict them.

Second, there are more or less bugs and debug interface in Microsoft Windows [9], which provides APIs to access the address space of other processes as well. So it is of great difficulty to prevent commercial software from being cracked.

Contemporary, existing anti-debugging methods can be divided into two categories. One is checking hardware or software debug register for special value which debuggers may use to place breakpoints on processes. Another method is to detect human behaviors such as unexpected pause in execution [1].

As a protection mechanism based on such hardware architecture and operation system, anti-debugging technology has limitations inevitably. First, each anti-debugging mechanism is designed for a specific debugging technique. It is probably the case that one anti-debugging tools targets WinDbg and it will definitely lose its power when a malware does not use WinDbg as debugging tool. Second, if malware or debuggers perceive that they are under attack from anti-debugging mechanism, reverse engineers will attempt to circumvent that attack or even strike back. Most of current anti-debugging mechanisms lack a transparent shield for hiding themselves from other processes’ view.

The growing popularity of hardware virtualization motivates our new solution for software protection through anti-debugging. Virtual machines can be an effective environment in which to prevent software from attacks and malicious behaviors[2][22]. Furthermore, taking advantage of hardware virtualization support, hypervisor is entitled to occupy a higher privilege level, termed root mode, than traditional kernel space to observe and protect target applications.

To the best of our knowledge, this paper is the first to address the issue of anti-debugging based on hardware virtualization in real-world application. As we will demonstrate, instead of relying on kernel space resource as conventional method did to construct anti-debugging architecture, we make use of advanced hardware virtualization technology to occupy higher privilege level so that debugging behavior could not avoid our protection shell trickily.

In this paper, we identify the typical features of debugging and discuss our design against them. Specifically, we make the following contributions.

  • We present and implement a lightweight and extensible framework called Spad to perform anti-debugging strategy transparently. The Spad hypervisor requires no code modification to the existing OS and target applications. As experiments reveal, our approach has successfully rendered most canonical debuggers incapable.
  • We implement a transparent memory-protecting mechanism which takes advantage of hardware virtualization and Memory-Hiding technology offering protection to hypervisor by memory remapping, thus makes it feasible to conceal the hypervisor in a private memory region.
  • We propose a general debugging model to describe its features and requirements. Based on such model, we separate each debugging component, interface and operation flow. Such splitting makes our solution reasonable and complete.

The rest of the paper is organized as follows. Section 2 presents the debugging model. Section 3 illustrates the architecture of our anti-debugging solution, while Section 4 describes the implementation details. Section 5 evaluates Spad through experiments. We survey related work in Section 6, then conclude in Section 7.

2. Debugging Model

To construct anti-debugging architecture, it is crucial to first understand that a debugger is not a solitary and independent system. Rather, debugging systems are composed of multiple, layered debugging subsystems that collectively facilitate debugging.

In this section, we describe the typical debugging model utilized in most debuggers. We use this model to generalize the features and requirements of debugging, which drive the design and architecture of our anti-debugging approach. Our model description is mainly established on debugging procedure. The statement starts by illustrating the layered debugging model, and then raises several basic assumptions for our anti-debugging design.

2.1 Debugging Layered Model

Debugging is the analysis of computer program which is performed by executing programs built from that software system on a real or virtual processor. To be effective, dynamic program analysis requires the target program to be executed with sufficient test inputs to produce meaningful behavior. Use of software testing techniques such as code coverage helps ensure that an adequate slice of the program's set of possible behaviors has been observed. Also, care must be taken to minimize the instrumentation effect on the execution (including temporal properties) of the target program.

Debugging Model.png
Figure # Debugging Layered Model

As shown in Figure #, the debugging model is divided into several layers. Each layer furnishes different level of abstraction of the debugging procedure. Details on each layer are listed as following.

Source-level debugger The starting point of the debugger layered model is the source-level debugger that executes on the target platform. Different target architectures include or support different source-level debuggers that allow extraction of runtime information over proprietary interfaces. The driver layer unifies these source-level debuggers as far as possible to provide various services for debugging procedure.

Driver components Most of debugging functions and libraries exist as driver components. Drivers generally implement only a narrow set of the interfaces, restricting the capabilities that the debugging system can offer to the higher levels. Common interfaces include run-control (starting, stopping, resetting, and stepping), access to symbol data (memory addresses of methods), and expression evaluation (reading variable values). Hardware-based debuggers also offer an interface, by default, for acquiring and reading real-time trace data from the target.

Viewpoint Viewed from the topmost layer, the model offers the user different viewpoints of the debugger. A viewpoint defines a specific view of the system and mediates between the target system and the visualization layer. The complete set of viewpoints offers insight to all aspects of the running program.

Debugger core Between viewpoint and driver is the debugger core composed of mapper and controller that bridge the layers. The viewpoint uses the controller to control the system from the model’s viewpoint. An execution step in a statechart corresponds to a complex sequence of commands at the source-code level. The mapper updates the runtime information model that provides the basis for the visualization with information from the source-level debug drivers.

Visualization Most current mainstream debugging engines, such as gdb and dbx provide console-based command line interfaces. Debugger front-ends are popular extensions to debugger engines that provide IDE integration, program animation and visualization features, which are implemented on the visualization layer.

2.2 Debugger classification

Debugging medium could be diverse. As the computer system evolve to such a complex architecture, debugger developers make use of every accessible resources, both hardware and software, to implement their effective debugging mechanism. In view of the difference of debugging mechanism, we classify all prevailing debuggers into four categories: Ring3 level debugger, Ring0 level debugger, native debugger and simulation debugger.

Ring3 Level Debugger

Ring0 Level Dbugger

Native Debugger

Simulation Debugger

2.2 Basic Assumptions

Some debuggers have the ability to modify the state of the program while it is running, rather than merely to observe it. It may also be possible to continue execution at a different location in the program. In other words, debug is not only a method for program observation, but also a probable way via which user can manipulate the runtime application.

This paper is concerned with the runtime debugging of software in both kernel space and user space. In order to focus on this problem, we assume that debugging tools could share the same privilege level as kernel space and not higher, which albeit some hardware may provide. Meanwhile, given that our anti-debugging solution is based on hardware virtualization, we also make the assumption that the target hardware platform support virtualization technology such as Intel virtual machine extension.

3. Architecture

In this section, the architecture of our anti-debugging system, called Spad will be introduced. As the debug model discussed before, a lightweight, transparent and reliable anti-debugging system needs to be implemented.

  • First, our system will be implemented based on hardware virtualization. Such technology divides the state of machine into root model and non-root model, the guest OS runs in the non-root model as the VMM runs in the root model. The VMM can monitor the behavior of guest OS. While a being monitored behavior happened in the non-root state, a VMEXIT will happen and change the state of machine into root-model, and then the corresponding trap will be achieved. For the trap is accomplished in the root-model, the guest OS runs in the non-root model could not detect the procedure. Thus the transparency to the guest OS is guaranteed and also there is no modification need to be done in the guest OS.
  • Second, among available software can utilize hardware virtualization, we realized a hypervisor which tailored from BluePill as the base for implementing our system. We transplant it from 64-bit to 32 -bit. It’s a tiny, lightweight, reliable framework for our Spad, and it is easily extensible and configurable.
  • Third, we select the Intel VT platform as our hardware virtualization platform.

Figure * shows the Spad’s architecture. There is no extra- component run in the Guest OS. And there are three modules in the hypervisor: memory-hidden module, filter module and anti-debugging module.

  • Memory-Hidden module: for our Spad is installed on the fly, to realize a complete transparency for Spad to the guest OS is essential, this module take a memory-hidden strategy to cover the Spad.
  • Filter module: this module is responsible for checking the identity of the trapped application. All the protected applications will be recorded in the hypervisor. While a debug action happens, the Spad will enter the root-model and check the application caused model shift. If the application is in the list, the module will transfer the control to the fist module. Else, the normal instruction stream of the trapped application will be executed.
  • Anti-debug module: while the shield module monitored the protected application is being debug, which means there is another application is debugging the protected application, this module will make the debug lose efficacy.

Spad Architecture.png

4. Implementation

This section describes the implementation of our anti-debugging system called Spad. We will introduce hypervisor,memory-hidden module, exception capture, shield module and fist module in turn.

  • We use a light-weight hardware virtualization based hypervisor to bear our anti-debugger framework. As the hypervisor enables us to capture some debugger's special actions, including generating of INT1 and INT3, CR3 conversion, and so on. All the actions above can be treated by Spad.
  • In our current implementation, the light-weight hypervisor realizes some features, such as install/uninstall on the fly, flexible configuration, and memory-hiding. Install/Uninstall on the fly inverts traditional process which is advised by virtualization technology to start the hypervisor before constructing and powering on each virtual machine. By install/uninstall on the fly, implemented by storing the context of guest OS, the hypervisor can be easily used and debugged. This light-weight hypervisor also supplies a rich set of interfaces to configure Virtual Machine Control Structure (VMCS) and memory strategy. With the help of flexible configuration, we could support various ways to protect software.

Control Flow

  • Considering a single guest OS running on the hypervisor which is based on hardware virtualization technology, the whole system have only one context among the following three types at any time: guest application, guest kernel, and hypervisor. While the hypervisor interested or sensitive events happened in the guest OS, the system will shift mode from non-root to root and the hypervisor will handle the events. Later, hypervisor will transfer control back to the guest OS after it finish handling the events.
  • Figure * shows that the instruction either in the guest user mode or in the kernel mode are capable to trigger hardware to generate VMEXIT event and transfer control to the hypervisor. For example, while a CPUID instruction is executed, a VMEXIT event is generated and hardware activates the hypervisor automatically to handle the event (transition 1). The hypervisor examines the stored guest OS state and its event strategy to determine how to handle it, and then the VMX instruction will be used to transfer the control to guest OS, resume the execution of guest OS.
  • In fact, guest kernel has more chance to cause mode shift. If the hypervisor is configured to monitor the RDMSR instruction, the guest OS will always triggers VMEXIT event once to read MSR registers. Then the hypervisor will do the same handling process as the applications.(transitions 3,4)
  • Our hypervisor also supports interpreting the guest OS on accessing physical resources such I/O related instructions. A hypervisor could always perform additional operations on the I/O access(transition 5,6) before resuming to the guest OS. With such approach, our hypervisor could cheat the guest OS to protect applications.

Memory-Hidden module

  • For the hypervisor is installed on the fly, a hypervisor is vulnerable if it can be accessed from guest applications or even guest OS. So, the hypervisor should keep transparent to the guest application and guest OS. Memory-Hiding technology is applied to conceal the hypervisor completely. With this technology, the hypervisor could maintenance transparency to guest OS.
  • Now in this section, let's talk about the details of this technology. A typical Operation System needs to build and manage process page tables for address translation. When the hypervisor was created, OS will build an address mapping for it. For loading all data and codes of the hypervisor by guest OS, a mapping from the hypervisor’s virtual address (VA) to real physical address (PAreal) is created as P(VA, PAreal). So the whole hypervisor can be easily accessed by the OS kernel. That is threat for our hypervisor and feature of install/uninstall on the fly. What's more, being accessible from the guest OS, a hypervisor can be easily modified and unloaded in the face of a malicious kernel. Figure# shows...

No memory hiding.png

  • In order to implement the memory-hiding technology, main method is to modify the corresponding PTE in the page table of guest OS. First, we clone page table of current guest OS as private usage of hypervisor, and then change the mapping of memory space from P(VA, PAreal) in the hypervisor own address space to P0(VA, PAspare), where PAspare refers to the physical address of a special spare page which not be used. This strategy makes all mapping to the hypervisor swept out from page table of guest OS. When switch context to execute hypervisor, the private page table takes effect and the hypervisor can reference itself. As shown in Figure #...

Memory hiding.png
(Spad's anti-debugging)

Exception capture

  • As we know, the method used in debugging is single step and breakpoint. Both of them need to cause INT1 and INT3 exception. In this section, we will divide two parts to describe concrete realization of Spad's anti-debugging module. The first is software breakpoint and anti-debugging (INT3). And the second is hardware breakpoint and anti-debugging (INT1).

(software breakpoint and anti-debugging)

  • For Intel x86 architecture, CPU provides an interrupt instruction "INT3" in protected-mode. This instruction could trigger a breakpoint exception, which can break down the running sequence on the fly and carry the interrupt handling routine of exception into execution. Then OS may transfer the control flow in the interrupt handler to debugger to analyze the breakpoint. When we use a debugger such as Visual Studio and WinDBG to set a breakpoint to a certain line of code, the debugger will substitute the first byte of the instruction of this line with "0xCC"(which is the machine code of "INT3"). And the code which is replaced is stored by the debugger. When code is executed at this line, the instruction "INT3" will be executed instead of the original instruction and the breakpoint exception will be triggered. Then the instruction pointer will jump to the corresponding interrupt handling routine, which now points to the debugger application. After the program falls into the debugger application, the previously changed byte will be recovered, and the instruction pointer will point to the interrupted instruction which could be correctly executed at present.

(hardware breakpoint and anti-debugging)

  • IA32 CPU has eight debug registers and the low four registers (DR0~DR3) store the linear address of the breakpoint. CPU always compares the current IP value with that linear address. Once the value is the same, "INT1" exception will be triggered and the instruction pointer will jump to the corresponding exception handler. Then a debugger application will run. All the following procedure is similar to "INT3".
  • On the other hand, "INT1" exception could also be used as single-step interrupt which is caused by the internal state of machine. When TF flag of the system flag register is enabled, single-step interrupt will be triggered automatically and make CPU pause by executing just one instruction. In the interrupt handling routine, the TF flag will be disabled which can be restored to be set at the end of the routine. In order to trigger the "INT1" exception in the next time.
  • So our mechanism of software debugging is to intercept the exception via VMM based on hardware virtualization technology.
  • Exceptions cause VMEXIT based on the exception bitmap. The exception bitmap is a 32-bit field that contains one bit for each exception. If an exception occurs, its vector is used to select a bit in the exception bitmap. If the bit is 1, a VMEXIT occurs; if the bit is 0, the exception is delivered normally through the guest IDT.
  • Thus, we set the corresponding bit in the exception bitmap to 1, so that each INT1 and INT3 exception will cause VMEXIT and enter the hypervisor to do further action.

Filter Module

  • In the guest OS, may be other application but not the protected ones will cause the two sensitive exception. On the other hand, Protecting all processes and software makes a great overload. And our original intention is to take Spad to protect certain software. So, a filter to distinguish protected ones from others is necessary. By all these reasons, Spad uses a filter module to protect software which we want to save.
  • In our implementation, we get the PID of the processes which we choose to protect and save in a white list in the hypervisor. When a debug exception is captured, we get the PID of the tapped process and compare to the PIDs listed in the white list to judge whether this process should be protected. While the filter module found the trapped process need to be protected, it will notify the anti-debugging module to do the following actions. But to the INT1 exception, we need to treat it conditionly for there are other interrupt vector 1 conditions, such as Data Memory and I/O Breakpoint Exception Conditions, General-Detect Exception Condition and Task-Switch Exception Condition. To this situation, we can distinguish the single-step by checking BS bit.

Filter Module.png

  • We can see the process of this strategy in the figure #. When software is being debugged, Spad hypervisor could cut down the debugged process by "INT1" exception and "INT3" exception through hardware virtualization technology. Then the filter module collects the message of the debugged software, and get PID value from the PCB block of this process or software. The next, we compare the PID value of this process with the saved one to judge whether this process need to be protected. And the saved PID is stored when the process is created. After comparison, if the PID value is same with that of protected process, Spad will check BS bit of DR6 register when interrupt vector 1 is triggered. If BS bit is disabled, Spad will run the protect module and stop this debugging action. However, if PID value is different or BS bit is enabled, Spad could let the process resume the next and do nothing.

Anti-Debug Module

  • After it received the notification from shield module, this module will interpret the exception to make it lose efficacy or even terminate the trapped process. While the guest OS transfer the control to the hypervisor, the length of the trapped instruction will be record as well as the instruction pointer. Generally, jump to the next instruction and do nothing is the simplest way. Even, we can call the kernel’s function to terminate the process, but may cause unexpected influence if the guest kernel is malicious.

5. Evaluation

Our experiments were conducted on a desktop computer with an Intel Core2 Duo E6320 processor and 2GB RAM. SPAD is installed on both the processor cores. Windows XP SP2 is selected as the guest operating system of the testbed.

Anti-Debug Ability Test Figure 1 shows the ability test result of our anti-debug approach. In this experiment, we select 13 anti-debug mechanisms , which include most of general anti-debug tricks against debuggers on Windows platform. We group these anti-debug tricks into 3 categories:

  • API The most straightforward way to detect the presence or the operation of a debugger is to use Windows provided functions, which can be either documented or undocumented and exported in various DLL and SYS files.
    • Invoking IsDebuggerPresent IsDebuggerPresent() function is supplied in Windows APIs to tell the calling process whether it is being debugged by a user-mode debugger. As the most common debugger-detecting method, an anti-anti debugger would like to hook this function or add a software breakpoint at its entry.
    • Check Remote Debugger Exported from kernel32.dll, CheckRemoteDebuggerPresent() function is also used to detect the presence of debugger in the current process. Similar to IsDebuggerPresent() function, this Windows API is also most likely to be hooked by any anti-anti debugger method.
    • Test SeDebugPrivilege A process will have full control on CSRSS.EXE once it gets the SeDebugPrivilege privilege, and this privilege will be inherited to its child processes. As a result, if one process has this privilege, it is probably under debugging and gets the privilege from the debugger at creation.
    • Check Parent Process This mechanism determines debugging by simply check its parent process. Normally, a user executes applications by double clicking the shortcut or by command-line, so the parent process of the application should be either EXPLORER.EXE or CMD.EXE on Windows XP. Thus, if the parent process is something else, it possibly indicates the application is running in a debugging environment.
    • Find Debugger Window By iterating the running application’s window handles, it is easy to find a running debugger. However, this anti-debug trick can only tell the existence of debugger instead of reliably pointing out the current process is debugging.
    • CloseHandle() Exception Detection Passing in an invalid handle to the CloseHandle() function will trigger an EXCEPTION_INVALID_HANDLE (0xc0000008) exception in the case of current process debugging. However, this will not happen in normal execution, which will return an error code as result.
    • OutputDebugString()'s LastError If the current process is running under a debugger, calling GetLastError() function next to OutputDebugString() function will get 0 as the return value, but it is not true without a debugger. Using this discrepancy it is possible to detect if a process is running inside a debugger.
  • Structure Another way to detect debugger is to traverse and analyze Windows kernel structure. According to Windows System Error Handling mechanisms, when a process is debugging, certain structures are modified to enable dispatching debug messages to the debugger.
    • Checking BeingDebug Flag in PEB The BeingDebug flag in PEB is set when the current process is under debugging. Thus an anti-debugger can check this value to detect the debugger.
    • Check Debug Port Check debug port can find out the debugger debugging the current process. In Windows, the process debug port is set to the Windows subsystem’s general function port in the condition of debugging the current process. This setting ensures that related debug events route to the process’s debugger.
    • Check Debug Object Handle Start from Windows XP and Windows 2003, debug object is introduced in to play the same role as debug port. As a result, checking debug object handle can also tell if the current process is debugging.
  • Exception In the case that a process is debugging, the debugger will overtake the exceptions of debuggee process. Further more, the debugger is not likely to hand these exception back to the debugging process. Thus, by building a well-designed exception trap, a process is able to determine whether it is debugging or not.
    • INT3 Exception Detection In the case when an INT3 triggered and no debugger is present, an exception is dispatched to the original process. Usually it is expected to be caught and handled in the appropriate CATCH statement block. However, if the process is running within a debugger, the debugger captures the interrupt and never hands it back. As a result, it changes the instruction flow of the debugging process thus it is possible to detect the process is debugging.
    • Single Step Exception Detection Similar to the INT3 interrupt, a running process can set TF in EFLAGS/RFLAGS register to enable single step trap. Thus it is also possible to use this method to detect debuggers.
    • INT2D Exception Detection INT 2D is also used in debugging. The trap handler for “INT 2D” constructs an EXCEPTION_RECORD structure with an exception code of STATUS_BREAKPOINT, and then hands it over to the kernel debugger.


Figure 1. The Anti-Debug Ability Comparison under Famous Debuggers. 13 general anti-debugger mechanisms are picked up to compare their effectiveness with our approach.

In this experiment, the result exhibits SPAD is effective against traditional debuggers. However, it shows no effect at the presence of ROR debugger. The reason is that ROR debugger builds up a software virtual machine and executes the debuggee application in a sandbox. Thus no INT3 or INT1 call can be detected in SPAD externally while the ROR debugger is working. (Future work或者related work讲下SPAD对于基于纯软件模拟的调试器无效)

Exp2.PNG Figure 2. The Isolation Test Result of SPAD.

Isolation Test In this experiment we test the correctness of SPAD by checking the result of packed bzip2 and original bzip2. Six famous shells with certain level anti-crack protection are selected. Since most of the six shells have software anti-debug tricks, we can test whether they can coexist with SPAD. As shown in figure 2, most of these shells with various software anti-debug tricks have no conflicts with SPAD. But some of them introduce in multi-process protection. Its mechanism is to fork a guard process of the packed application, acting like a debugger in order to fight against other real debuggers from attaching. SPAD may fail in such situations since it can not tell the guard process (which behaviors similar as a debugger) from real debuggers.


6. Related Works

Understanding the execution behavior of malicious programs is of critical importance. Compared to static malware analysis [22], dynamic analysis of malware programs gives rich information about runtime behaviors. Recent work performs anti-debugging strategy for software protection is based on behavioral patterns. For example, behavior-based clustering of malware programs was proposed [2]. Academic research and industry have rising much attention on virtual machine technology. On the other hand, priors have been making every endeavor to design anti-debugging tools, albeit not very executable. Although it is known that both virtualization technology and conventional anti-debugging methods exist, to the best of our knowledge, no existing work has extensively studied for the technology immerging of them. As mentioned, our prototype design and implementation are advanced in software protection through anti-debugging with hardware virtualization technology.

Hardware virtualization technology enables transparent intercepting guest OS exceptions and interrupts. One example is Overshadow [5], which also can be implemented by HBSP. With the help of additional layer of address translation, it provides different views of the sensitive application’s address space according to the current context. Without affecting the existing OS and legacy protected application, even the hardware, kernel and other programs can only get the encrypted content from the protected application’s virtual space.

Intel’s Trusted Execution Technology (TxT)[25] is another hardware-based software security approach, providing isolated protected execution environment, which offers no privilege to unauthorized software even to observe. Furthermore, it provides protected input and storage channel to ensure the data security. This approach can be regarded as a complement of our Memory-Hiding technology, albeit only available on some special hardware.

7. Conclusion

Debugging is a widespread method for runtime observation and manipulation of application program. This method is useful and helpful to software development, but it may also be wicked if malicious code or attackers employ it. This paper presents a lightweight and transparent protection strategy based on the art of hardware virtualization. A tiny hypervisor is introduced to intercept debugging behavior and fight back actively. We have done plenty horizontal experiments with other anti-debugging software and vertical experiments with several popular and canonical debuggers. The results show that our anti-debugging solution is effective to disable the functionalities of most debuggers as well as making them imperceptible. To the best of our knowledge, this is the first work to address anti-debugging problem with hardware virtualization technology.


We would like to thank the anonymous reviewers for their insightful comments and suggestions. Our work is supported in part by National Natural Science Foundation of China (Grant No.60773093 and 60873209), the Key Program for Basic Research of Shanghai (Grant No.09JC1407900), Microsoft Research Asia Young Teacher Funding and Intel Corporation.


Software Protection through Anti-Debugging
Gagnon, M.N.; Taylor, S.; Ghosh, A.K.;
Security & Privacy, IEEE Volume 5, Issue 3, May-June 2007 Page(s):82 - 84

Towards an understanding of anti-virtualization and anti-debugging behavior in modern malware
Xu Chen; Andersen, J.; Mao, Z.M.; Bailey, M.; Nazario, J.;
Dependable Systems and Networks With FTCS and DCC, 2008. DSN 2008. IEEE International Conference on
24-27 June 2008 Page(s):177 - 186

Traps, events, emulation, and enforcement: managing the yin and yang of virtualization-based security
Sergey Bratus, Michael E. Locasto, Ashwin Ramaswamy, Sean W. Smith
October 2008 VMSec '08: Proceedings of the 1st ACM workshop on Virtual machine security

Virtualization-assisted framework for prevention of software vulnerability based security attacks
Aaraj, N., Raghunathan, A. and Jha, N.K.
Technical Report CE-J07-001, Dept. of Electrical Engineering, Princeton University. 2007

"Revisiting Software Protection,"
P.C. van Oorschot,
Proc. 6th Int'l Conf. Information Security (ISC 03), LNCS 2851, Springer-Verlag, 2003, pp. 1–13;

Policy Enforcement and Compliance Proofs for Xen Virtual Machines
Bernhard Jansen (IBM Zurich Research Laboratory), HariGovind Ramasamy (IBM T. J. Watson Research Center) and Matthias Schunter (IBM Zurich Research Laboratory)
VEE '08: Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Opening Black Boxes: Using Semantic Information to Combat Virtual Machine Image Sprawl ;
Darrell Reimer (IBM T. J. Watson Research Center), Arun Thomas (University of Virginia), Glenn Ammons, Todd Mummert, Bowen Alpern and Vasanth Bala (IBM T. J. Watson Research Center) [slides]
VEE '08: Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Execution replay for multiprocessor virtual machines ;
George Dunlap, Dominic Lucchetti (University of Michigan), Michael Fetterman (University of Cambridge) and Peter Chen (University of Michigan) [slides]
VEE '08: Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Automated detection of persistent kernel control-flow attacks
Nick L. Petroni, Jr., Michael Hicks
October 2007 CCS '07: Proceedings of the 14th ACM conference on Computer and communications security

Multi-aspect profiling of kernel rootkit behavior
Riley, Ryan and Jiang, Xuxian and Xu, Dongyan
EuroSys '09: Proceedings of the 4th ACM European conference on Computer systems

Instruction-level countermeasures against stack-based buffer overflow attacks
Francesco Gadaleta, Yves Younan, Bart Jacobs, Wouter Joosen, Erik De Neve, Nils Beosier
March 2009 VDTS '09: Proceedings of the 1st EuroSys Workshop on Virtualization Technology for Dependable Systems

Extending virtualization services with trust guarantees via behavioral monitoring
Himanshu Raj, Karsten Schwan
March 2009 VDTS '09: Proceedings of the 1st EuroSys Workshop on Virtualization Technology for Dependable Systems

Switchblade: enforcing dynamic personalized system call models
Christof Fetzer, Martin Sü?kraut
April 2008 Eurosys '08: Proceedings of the 3rd ACM SIGOPS/EuroSys European Conference on Computer Systems 2008

DOC - answering the hidden 'call' of a virus.
E. U. Kumar, A. Kapoor, and A. Lakhotia.
Virus Bulletin, April 2005.

A method for detecting obfuscated calls in malicious binaries.
A. Lakhotia, E. U. Kumar, and M. Venable.
IEEE Transactions on Software Engineering , 31(11):955-968, 2005.

Emulated Breakpoint Debugger and Data Mining Using Detours
PJason Raber, PEric Laspe
Oct. 2007 Proceedings of the 14th Working Conference on Reverse Engineering

ExecRecorder: VM-based full-system replay for attack analysis and system recovery
Daniela A. S. de Oliveira, Jedidiah R. Crandall, Gary Wassermann, S. Felix Wu, Zhendong Su, Frederic T. Chong
Oct. 2006 Proceedings of the 1st workshop on Architectural and system support for improving software dependability

Debugging operating systems with time-traveling virtual machines
PSamuel T. King, PGeorge W. Dunlap, Peter M. Chen
Apr. 2005 Proceedings of the annual conference on USENIX Annual Technical Conference

Philipp Graf and Klaus D. Müller-Glaser,
Gaining Insight into Executable Models during Runtime:
Architecture and Mappings, IEEE Computer Society, March 2007 (vol. 8, no. 3).

Hybrid static-dynamic attacks against software protection mechanisms
Matias Madou,Bertrand Anckaert,Bjorn De Sutter,Koen De Bosschere
ACM Workshop On Digital Rights Management Proceedings of the 5th ACM workshop Digital rights management
Nov. 2005 - Nov. 2005, Alexandria, VA, USA

Mark E.R., D.A. Solomon,
Microsoft Windows Internals, 4th ed.,
USA: Microsoft Press, 2005

X. Jiang and D. Xu.
Collapsar: A VM-Based Architecture for Network Attack Detention Center.
In Proceedings of the 13th USENIX Security Symposium, August 2004.

Hacker Debugging Uncovered by Kris Kaspersky

C. Kr¨ugel, W. K. Robertson, F. Valeur, and G. Vigna. Static disassembly of obfuscated binaries. In USENIX Security Symposium, pages 255–270, 2004.

SoftICE (2008) 'SoftICE: kernel mode debugger for Microsoft Windows, its production is suspended', http://en.wikipedia.org/wiki/SoftICE.

Intel. Intel Trusted Execution Technology Architecture Overview. September 2006.

X. Chen, T. Garfinkel, E. C. Lewis, P. Subrahmanyam, C. A. Waldspurger, D. Boneh, J. Dwoskin and D. R.K. Ports. Overshadow: A Virtualization-Based Approach to Retrofitting Protection in Commodity Operating Systems. In \emph{ASPLOS’08}, March 2008.

Ether: malware analysis via hardware virtualization extensions Artem Dinaburg, Paul Royal, Monirul Sharif, Wenke Lee October 2008 CCS '08: Proceedings of the 15th ACM conference on Computer and communications security

Covert Debugging Circumventing Software Armoring Techniques D Quist, V Smith - Black Hat Briefings USA, 2007 - Citeseer

SPiKE: engineering malware analysis tools using unobtrusive binary-instrumentation
Amit Vasudevan, Ramesh Yerraballi
January 2006 ACSC '06: Proceedings of the 29th Australasian Computer Science Conference

Windows Anti-Debug Reference

Stealth Breakpoints
Amit Vasudevan, Ramesh Yerraballi
December 2005 ACSAC '05: Proceedings of the 21st Annual Computer Security Applications Conference

Hybrid static-dynamic attacks against software protection mechanisms
Matias Madou, Bertrand Anckaert, Bjorn De Sutter, Koen De Bosschere
November 2005 DRM '05: Proceedings of the 5th ACM workshop on Digital rights management

SecVisor: a tiny hypervisor to provide lifetime kernel code integrity for commodity OSes
Arvind Seshadri, Mark Luk, Ning Qu, Adrian Perrig
October 2007 SOSP '07: Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles

A hypervisor-based system for protecting software runtime memory and persistent storage
Prashant Dewan, David Durham, Hormuzd Khosravi, Men Long, Gayathri Nagabhushan
April 2008 SpringSim '08: Proceedings of the 2008 Spring simulation multiconference

Using hypervisor to provide data secrecy for user applications on a per-page basis
Jisoo Yang, Kang G. Shin
March 2008 VEE '08: Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Efficiently tracking application interactions using lightweight virtualization
Yih Huang, Angelos Stavrou, Anup K. Ghosh, Sushil Jajodia
October 2008 VMSec '08: Proceedings of the 1st ACM workshop on Virtual machine security

Dusseau. “VMM-based hidden process detection and
identification using Lycosid,” In ACM Conference on
Virtual Execution Environments (VEE), Seattle, WA, Mar.2008.

X. Jiang, X. Wang, and D. Xu.
“Stealthy malware detection through VMM-based ‘out-of-the-box’ semantic
view,” In 14th ACM Conference on Computer and Communications Security (CCS), Alexandria, VA, Nov. 2007.

Asrigo, K., Litty, L., and Lie, D.
“Using VMM-based sensors to monitor honeypots,”
In Proceedings of the 2nd international Conference on Virtual Execution Environments, Ottawa, Ontario, Canada, June 2006).

40 A tracer driver for hybrid execution analyses
Ludovic Langevine, Mireille Ducassé
September 2005 AADEBUG'05: Proceedings of the sixth international symposium on Automated analysis-driven debugging Publisher: ACM

Digital watermarking security considerations
Rade Petrovic, Babak Tehranchi, Joseph M. Winograd
September 2006 MM&Sec '06: Proceedings of the 8th workshop on Multimedia and security

Yuschuk, O. (2008) 'OllyDbg: Win32 debugger', http://www.ollydbg.de/.

Black Fenix. Black fenix's anti-debugging tricks. http://in.fortunecity.com/ skyscraper/browser/12/sicedete.html.

S. Cesare. Linux anti-debugging techniques (fooling the debugger), January 1999. VX Heavens. http://vx.netlux.org/lib/vsc04.html.

Personal tools
Upload file