On October 2nd, 2023, the BeyondTrust security teams detected an identity-centric attack on an in-house Okta administrator account. We immediately detected and remediated the attack through our own Identity Security tools, resulting in no impact or exposure to BeyondTrust’s infrastructure or to our customers. The incident was the result of Okta’s support system being compromised which allowed an attacker to access sensitive files uploaded by their customers.

What happened?

The incident began when BeyondTrust security teams detected an attacker trying to access an in-house Okta administrator account using a valid session cookie stolen from Okta’s support system. Custom policy controls blocked the attacker's initial activity, but limitations in Okta's security model allowed them to perform a few confined actions.

The initial incident response indicated a possible compromise at Okta of either someone on their support team or someone in position to access customer support-related data. BeyondTrust raised their concerns of a breach to Okta on October 2nd. Having received no acknowledgement from Okta of a possible breach, BeyondTrust persisted with escalations within Okta until October 19th when Okta security leadership notified them that they had indeed experienced a breach and BeyongTrust were one of their affected customers.

Okta Security has identified adversarial activity that leveraged access to a stolen credential to access Okta's support case management system.

Within the course of normal business, Okta support will ask customers to upload an HTTP Archive (HAR) file, which allows for troubleshooting of issues by replicating browser activity. HAR files are HTTP archives that can be generated by a web browser to log interactions with a website, in this case used for debugging an issue with the site. The administrator complied with the request and generated a HAR file containing an API request and a session cookie which was uploaded to the Okta support portal.

HAR files can also contain sensitive data, including cookies and session tokens, that malicious actors can use to impersonate valid users.

Within 30 minutes of the administrator uploading the file to Okta’s support portal an attacker used the session cookie from this support ticket, attempting to perform actions in the BeyondTrust Okta environment. BeyondTrust’s custom policies around admin console access initially blocked them, but they pivoted to using admin API actions authenticated with the stolen session cookie. API actions cannot be protected by policies in the same way as actual admin console access. Using the API, they created a backdoor user account using a naming convention like existing service accounts.

In an interview with KrebsOnSecurity, Okta’s Deputy Chief Information Security Officer Charlotte Wylie said Okta initially believed that BeyondTrust’s alert on Oct. 2 was not a result of a breach in its systems. But she said that by Oct. 17, the company had identified and contained the incident — disabling the compromised customer case management account, and invalidating Okta access tokens associated with that account.

Wylie declined to say exactly how many customers received alerts of a potential security issue, but characterized it as a “very, very small subset” of its more than 18,000 customers.

A backstory that is important to remember regarding Okta is that early 2023, Okta laid off 5% of the workforce.

Previous Issues

In March 2022, Okta disclosed a breach from the hacking group LAPSUS$, a criminal hacking group that specialized in social-engineering employees at targeted companies. An after-action report from Okta on that incident found that LAPSUS$ had social engineered its way onto the workstation of a support engineer at Sitel, a third-party outsourcing company that had access to Okta resources.

Important Note

It should be noted that the Okta support case management system is entirely separate from the production Okta service, which is fully operational and has not been impacted. In addition, the Auth0/CIC case management system is not impacted by this incident.

Indicator of Compromise

The following IOC were discovered during the attack.

IP Addresses

23.105.182.19 104.251.211.122 202.59.10.100 162.210.194.35 (BROWSEC VPN) 198.16.66.124 (BROWSEC VPN) 198.16.66.156 (BROWSEC VPN) 198.16.70.28 (BROWSEC VPN) 198.16.74.203 (BROWSEC VPN) 198.16.74.204 (BROWSEC VPN) 198.16.74.205 (BROWSEC VPN) 198.98.49.203 (BROWSEC VPN) 2.56.164.52 (NEXUS PROXY) 207.244.71.82 (BROWSEC VPN) 207.244.71.84 (BROWSEC VPN) 207.244.89.161 (BROWSEC VPN) 207.244.89.162 (BROWSEC VPN) 23.106.249.52 (BROWSEC VPN) 23.106.56.11 (BROWSEC VPN) 23.106.56.21 (BROWSEC VPN) 23.106.56.36 (BROWSEC VPN) 23.106.56.37 (BROWSEC VPN) 23.106.56.38 (BROWSEC VPN) 23.106.56.54 (BROWSEC VPN)

User-Agents

While the following user-agents are legitimate, they may be rare in your environment given the release of Chrome 99 in March 2022.

Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.7113.93 Safari/537.36 (Legitimate, but older user-agent)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36 (Legitimate, but older user-agent)

Detection

According to Okta’s statement, the threat-actor accessed Okta’s customer support system and viewed files uploaded by certain Okta customers as part of recent support cases. It appears that in our case, the threat-actor was able to hijack a session token from a support ticket which was created by a Cloudflare employee. Using the token extracted from Okta, the threat-actor accessed Cloudflare systems on October 18. In this sophisticated attack, Cloudflare observed that threat-actors compromised two separate Cloudflare employee accounts within the Okta platform. Cloudflare detected this activity internally more than 24 hours before they were notified of the breach by Okta. Upon detection, Cloudflare SIRT was able to engage quickly to identify the complete scope of compromise and contain the security incident.

What made the difference was that detection made it possible to detect and mitigate this attack. It's noteworthy to mention that customers were detecting attack against Okta. And its regretful to mention that Okta did not take immediate action and took delayed action on this matter, after persistence from the customers. The same issue had happened with Microsoft with the E5 level logging.

Organizations that provide services needs to be better prepared and proactive. It's dissapointing to read that companies that sells security don't do security themselves.

Blue team points

Here are some detection hints that can be used specifically for the Okta as well as detection in general

• Session Hijacking - Look for suspicious essions appearing without an authentication event that are consistent with session hijacking
• Proxy Logins - check for proxy logins for administrative use (normal users wont be using proxies for admin purposes
• Privilege management - monitor for privilege escalation or granting of privilege to users or backdoor accounts.
• Password health report - this report should be generated when a suspicious activity is detected or suspected
• Using strong authentication - administrative accounts needs thorough protection - such as FIDO2 tokens for authentication. While this is the case, remember that session hijacking can bypass MFA as authentication is not triggered. Hence a robust session verification system/process is important.

Reference

[1] Zaman, S. (2023). How Cloudflare mitigated yet another Okta compromise. Retrieved from https://blog.cloudflare.com/how-cloudflare-mitigated-yet-another-okta-compromise/?ref=upstract.com

[2] Brian Krebs (2023). Hackers Stole Access Tokens from Okta’s Support Unit. Retrieved from https://krebsonsecurity.com/2023/10/hackers-stole-access-tokens-from-oktas-support-unit/

[3] BeyondTrust Discovers Breach of Okta Support Unit. (n.d.). Retrieved from https://www.beyondtrust.com/blog/entry/okta-support-unit-breach

[4] Kapko, M. (2023). Okta CEO blames overhiring, “execution challenges” for layoffs. Retrieved from https://www.cybersecuritydive.com/news/okta-ceo-plans-layoffs/641919/