A Backdoored LiteLLM Release Passed Its Own Approval Gate, Signed Off With Stolen Tokens

Most supply chain compromises are described as a failure of trust in someone else's code. The LiteLLM incident on October 21 reads more like a failure of a process that worked exactly as configured. The attacker did not slip a package past a registry or fool a reviewer. They ran LiteLLM's own release workflow, had it approved by a maintainer's account, and let the pipeline publish the result with build provenance that faithfully recorded a malicious fork as the source.

LiteLLM is an open source gateway that sits between applications and LLM providers, which makes it a store of exactly the credentials attackers want. Datadog Security Labs says the package sees about 23 million weekly downloads. The route to compromising it started with a single key that nobody revoked.

A Key Leaked in May, Used in October

LiteLLM's own reconstruction of the attack chain starts on May 27, when an opencode.json file containing the master key for the LiteLLM Agent Platform was committed to the public LiteLLM-Labs/lite-harness repository. The commit was force-pushed over, but the key itself was not revoked. That is 136 days of exposure before it was used.

On the evening of October 10 the attacker used that key against the Agent Platform API, and one call returned a GitHub CLI OAuth token belonging to one of the two owners of LiteLLM's GitHub organization. The token was not tied to a service account and was not scoped down. LiteLLM notes that GitHub CLI tokens are issued after two-factor authentication and are not subject to GitHub's reauthentication prompts for sensitive actions, so the attacker now had owner-level access to every repository.

Within the hour, workflows pushed to temporary branches in two LiteLLM repositories posted repository secrets to a webhook.site endpoint, including a second maintainer's GitHub token and a third maintainer's release token. Each run was deleted about two minutes after it ran. On October 12 the attacker repeated the trick across several private repositories and collected all of their secrets.

How the Release Gate Was Satisfied

On the morning of October 21 the attacker, acting as owner, changed the release source in a private release repository to point at a fork. A push using the second maintainer's token was rejected by branch protection at 10:08:29. Five seconds later the owner's token disabled the protection and force-pushed. The staging workflow then ran against Berri-AI/litellm, a lookalike that, as Datadog's analysis points out, differs from the legitimate BerriAI/litellm by a single hyphen.

The production gate was the last control, and it had been loosened. LiteLLM says release gating had been relaxed “to accommodate the temporary push for more frequent releases,” allowing any two engineers to trigger and approve a release. At 11:11 the second maintainer's token approved it. Seven litellm==1.107.0 wheels reached PyPI between 11:39 and 11:40, and the Docker Hub and GHCR images followed at 11:46. A third maintainer's token then deleted the staging and production run records, and the attacker reverted the main branch to cover the change.

“Key revocation did not occur for the master key.”

LiteLLM, in its attack chain post

Detection was fast by any normal standard. A community member opened an issue flagging the compromise at 12:06, LiteLLM began its response at 13:02, and PyPI quarantined the version by 13:23. The Docker tags were removed or repointed by 14:20. LiteLLM says the artifacts were live for about two hours, and Wiz, citing ClickPy data, puts downloads of the malicious version at approximately 49,000.

Why Two Hours Was Enough

The payload added three files, cook.pth, cook.py and sitecustomize.py, to the Python environment. Because Python processes .pth files and sitecustomize.py at interpreter start, Datadog says any Python process on the host triggers the stealer, whether or not LiteLLM is imported, and deleting the package directory does not remove the files. The collection list covers AWS, GCP and Azure credentials, Kubernetes, Vault and Terraform secrets, SSH keys, Git, npm and PyPI tokens, and API keys for AI tools. Wiz found it also queries the proxy database for up to 500 rows of stored upstream provider credentials.

The stolen material was used quickly. Wiz observed the attacker validating AWS instance role credentials about four and a half hours after exfiltration. A second component checked a GitHub dead drop every hour for a domain to download more code from, though it held only test values before GitHub removed it.

This is also not LiteLLM's first incident this year. The Hacker News reports the same cook.pth behavior was seen when LiteLLM was compromised during the TeamPCP campaign in March, and Datadog notes that no connection between the two incidents has been established. LiteLLM's director of security, Oliver Jensen, told the publication that all tokens involved have been revoked, that the organization was cut down to a single unaffected user as a temporary measure, and that trusted-publisher links to its GitHub organization were removed from public registries.

What Engineering and Security Leaders Should Do Now

• Check for 1.107.0 everywhere Python runs. Look for cook.pth, cook.py and sitecustomize.py in site-packages, remove them by hand, and pin to 1.106.2 by digest, including in container images pulled by the latest tag.
• Rotate from a clean machine. Treat every cloud key, developer token and LLM provider key reachable from an affected host as stolen, and revoke active sessions as well as the keys themselves.
• Revoke leaked keys, not just the commit. Force-pushing over a secret removes it from view, not from use. Any key that touched a public repository should be revoked the same day.
• Make release approval mean two people. A two-person rule that two stolen tokens can satisfy is a single point of failure. Require approvals from separate hardware-backed identities and alert when protection rules are changed.
• Scope and expire developer CLI tokens. Owner-level tokens that are not service-bound and skip reauthentication should not live in tools or logs where an API can return them.
• Verify the source in provenance. Signed provenance only helps if someone checks that the recorded repository is the real one, so policy should reject builds from anything other than the expected repository.

Every control LiteLLM had in place fired as designed: branch protection rejected a push, the release gate asked for approval, and provenance recorded the source. The attacker simply held enough legitimate credentials to answer each one, starting with a key that should have died in May.