While analyzing the latest version of Citadel (126.96.36.199) we were able to observe two changes that try to make malware analysts' life harder. These changes also had been announced on a particular underground forum before they appeared in the wild.
[+] Added anti-emulator, which allows you to protect your botnet from reversing and getting into trackers. When it starts, a built-in detective checks if it is running in a virtual machine or in sandboxed environment (CWSandbox, VMware, Virtualbox), and if it is the case, it starts to behave differently and your botnet go unnoticed. Details were not disclosed, and the technology is very tricky.
Although they did not disclose any specific details about how the so called detection actually works, we could inspect it a bit further. It simply scans through the resources of the currently running processes and looks for specific patterns for instance inside the "CompanyName" field, such like:
Nevertheless, the tricky part comes here. When a virtualized environment detected, unlike many other Trojans that stop to work, Citadel will continue to operate, but behaves in a different manner. It will generate a unique-machine dependent domain name (obviously fake) and tries to connect to this server (unsuccessfully), making it to believe that the bot is dead and its command and control server is offline, meanwhile the real C&C domain is kept hidden. You can distinguish between the fake ones because the way they are generated, they look like an md5 hash itself, the C-style format string used is:
If we run a Citadel sample of this kind in a VMWare environment, closing all processes related to VMware (vmwareuser.exe, vmwaretray.exe, ...) will be enough to force Citadel to act normally as if it were running in a physical machine.
Instead of showing Olly or Ida screenshots, we are going to take a different approach. Let's take a dummy application like notepad and replace the company name with a resource editor such like Resource Hacker.
Doing this, if we run notepad, it will result in a fake infection and the malware will create the fake domain (first DNS query in the image), and if we run the Citadel sample without the altered version of notepad we are going to connect to the real C&C (second DNS query).
The another change that has an impact is the slightly modified RC4 algorithm. In altered one, the malware involves an internal "hash" within the algorithm.
While computing the stream cipher, in addition to the normal XOR operations of RC4, in each iteration the value is XORed with hash string's characters in a consecutive way.
The change in the RC4 algorithm affects also how the Trojan communicates with its control panel, due to the same algorithm is used to encrypt network traffic. Therefore the new control panel won't be able to handle connections coming from older versions of the bot.
Mikel Gastesi & Jozsef Gegeny