Anti-VM/Anti-Sandbox: Online Malware Sandbox Evaluation
Introduction
Malware usually sniffs the virtual sandbox environment to avoid automated behavior analysis, such as identifying unique files and processes of the virtual machine, or performing operations that the virtual machine cannot fully simulate. Therefore, for an automated behavior analysis system, how to disguise the virtual environment without being detected by malware is a very important task. This test selected 10 online malware sandbox analysis platforms to test and evaluate their simulation/disguise of the analysis environment.
The test used al-khaser, a tool similar to Pafish but with more updated anti-debugging, anti-sandbox, and anti-virtual machine detection technologies.
Test Content
Debugger Detection
- IsDebuggerPresent API, PEB.BeingDebugged…
- TLS callback
- Hard/software breakpoints
- VirtualAlloc
- …
DLL Injection Detection
- Detect whether there is a DLL injected into the process to monitor the behavior of the process (HOOK)
Virtual Box Detection
- Files and registries, including VBoxMouse.sys, VirtualBox Guest Additions directory…
- Check the registry, process, service, such as VBoxControl.exe, VBoxService…
- Detect hardware name, MAC address, etc…
- …
VMware Detection
- Similar to Virtual Box Detection
Other VMs Detection
- Xen, QEMU, Wine, Paralles…
Analysis Tool Process Detection
- Detect the existence of analysis tool processes such as OD, ProcessMonitor, Autorun…
General Sandbox/Virtual Machine Detection
- Detect whether there is a special process name, module name
- Detect real hardware status through WMI, such as hard disk size, memory size, CPU fan, model, BIOS serial number name, power supply voltage, temperature, etc…
*For more technical details of test items, please refer to al-khaser
*The original tool also has other testing methods, such as Timing Attack, but because timing attacks are hard to avoid and also difficult to test, they are not included in this test.
Test Vendor
| Vendor | Link |
|---|---|
| Threat Book (微步) | https://s.threatbook.cn/ |
| Tencent Habo (腾讯哈勃) | https://habo.qq.com/ |
| Maldun (魔盾) | https://www.maldun.com/analysis/ |
| Depthsec (微点) | https://sandbox.depthsec.com.cn/index.php/ |
| Qianxin (奇安信) | https://sandbox.ti.qianxin.com/sandbox/page |
| ANY.RUN | https://app.any.run/ |
| HybridAnalysis (Falcon Sandbox) | https://www.hybrid-analysis.com/ |
| CAPE Sandbox | https://capesandbox.com/ |
| BD Sandbox Analyzer | https://www.bitdefender.com/business/enterprise-products/sandbox-analyzer.html |
| Joe Sandbox | https://www.joesandbox.com/ |
Test Mark
* A total of 226 test points
“P” means that it passed, that is, it was not detected;
“F” means that it failed, that is, it was detected; “N/A” means that it crashed during the test, and no results are available.
Note: deleted some test points of the original tool that may have false positives or unstable.
Test Result
Summary
Sorted by pass rate, the higher the total percentage, the better. The green part is completely passed, the yellow part is part of the test items passed, and the red part is that no test items are passed.
Details
Conclusion
Obtaining the real hardware status through WMI is the hardest hit area… Almost no virtual machine can completely pass these test items.