Northflank Tells Every Customer to Rotate Secrets After a Breach, and Stolen AI Keys Are Already Being Used
Cloud hosting platform Northflank says customers should assume every environment variable on its platform has been decrypted, and some users have already seen their AI API keys abused. If your developers or vendors host apps there, here is what to rotate today.
If anyone building software for your business uses Northflank, today is a good day to change passwords and keys. The cloud application platform disclosed a security incident on October 30 and quickly widened its warning: every customer should assume that the environment variables stored on the platform have been decrypted and the secrets inside them taken.
The details below come from BleepingComputer’s reporting on the incident, which draws on Northflank’s public incident updates.
Where things stand
Northflank lets developers run applications, containers, databases and AI workloads without managing the servers underneath. Like most platforms of its kind, it stores configuration values called environment variables, which often hold database passwords, cloud credentials, access tokens and API keys. That is what makes this breach serious: a leaked environment variable is often a ready-made key to another service.
The damage is not hypothetical. Northflank confirmed that customers reported unauthorized use of AI API keys they had given to the platform, and it repeated its instruction to rotate any user-provided secrets that had not yet been changed.
How the first hours unfolded
- 11:43 UTC, October 30: cloud security company Wiz alerted Northflank to the threat. Engineers were looking at it a minute later.
- Shortly after: Northflank pushed a network policy patch.
- 12:50 and 13:05 UTC: two more patches followed, and the company brought in Wiz’s incident response team.
- Since then: Northflank turned on credential rotation for PostgreSQL, MongoDB, MySQL and Redis databases, and switched on audit logs for every customer, free accounts included, so they can look for suspicious activity.
Several important questions are still open. Northflank has not said how the attackers got in, how many customers were affected, or whether credentials other than AI keys were stolen or misused.
Part of a bigger pattern
Northflank is not alone. In research it calls “Black October,” Wiz counted at least nine security incidents at AI developer platforms during October, including Railway, OpenCode, Warp and Lightning AI. Several involved environment variables or AI keys. The goal appears to be what researchers call LLMJacking: using someone else’s AI keys to run models on their bill or to resell the access. In two cases Wiz looked into, stolen enterprise credentials drove roughly 2.5 million Amazon Bedrock API calls and about $100,000 in AI charges. Wiz has not said the nine incidents share one attacker.
Rotate today: a short list
- Ask your developers, contractors and software vendors whether any of your applications or data run on Northflank.
- If they do, replace every secret stored there, including those in workloads, secret groups and global secrets. Rotate at the source (the AI provider, database or cloud account), not only the copy on the platform.
- Use the platform’s new rotation feature for any hosted PostgreSQL, MongoDB, MySQL or Redis databases.
- Turn off public access to add-ons where it is enabled.
- Check usage and billing dashboards at your AI providers for spikes or activity you cannot explain, and report anything odd to both Northflank and the provider.
- Review the newly enabled audit logs for actions nobody on your team recognizes.
What this means for your company
Even if you have never heard of Northflank, the lesson applies. Modern apps run on a web of keys, and those keys usually sit with a hosting provider, a developer’s laptop or a build service. When one of those places is breached, the attacker does not need your password; they can simply use the key. Keep an inventory of which outside services hold your business’s secrets, set spending limits and usage alerts on AI and cloud accounts, give each key only the permissions it needs, and make sure you know how to rotate every key quickly, before you need to.
Not sure which of your apps, vendors or AI tools hold keys that could be used against you? Reach out to Layer 7 Data Solutions to discuss how we’ve helped companies similar to yours.
About The Author
