Skip to content
nuDefend
← Back to blog

Two LiteLLM security failures in one week: building an outbound allowlist for a server running an AI gateway

·12 min read

If you run LiteLLM or a similar gateway that routes requests to model providers, the server probably holds all the providers’ API keys, the proxy’s master key and the database password. Within one week, LiteLLM published details of two security failures. The first was a poisoned release distributed through the company’s release process. The second was a critical vulnerability in the proxy itself. LiteLLM’s guidance includes rotating keys and pinning versions. These steps are necessary, but they respond to an incident that has already happened. Before the next incident, it is worth asking a different question: where does the server need to connect, and what happens when it tries to connect somewhere else?

One week, three publications

21 October: a poisoned release. According to LiteLLM’s incident report, the company’s release infrastructure was compromised. The attacker used it to publish litellm==1.107.0 to PyPI and container registries. The release contained compiled malware that steals credentials and loads tools for remote code execution. The package was available on PyPI between 11:39 and 13:23 UTC. The Docker Hub tags v1.107.0, latest, main-latest and main-stable pointed to the infected image between 11:46 and 14:20 UTC. The company says anyone who installed an unpinned version or ran one of these tags during that window should assume their environment was compromised. Enterprise customers who pulled from the private registry were not affected.

23 October: the attack chain. The company’s root cause analysis supersedes the initial theories. On 27 May, a configuration file containing an admin key for an internal deployment of LiteLLM Agent Platform was added to a public repository. This is an experimental platform where cloud agents open pull requests. The file was deleted about three hours later, but human error meant the key was not revoked. On 10 October, the attacker used it, and the platform’s API returned the agents’ configurations. These included a GitHub CLI token belonging to one of the GitHub organization’s owners. The attacker used the token to run temporary workflows that sent CI secrets to an endpoint on webhook.site. The secrets included tokens belonging to two other maintainers. On 21 October, the attacker changed the release workflow to build from a repository in the similarly named organization Berri-AI. They approved deployment to production with one token and deleted the run records with another. According to the company, the attacker mainly searched other internal systems for model provider API tokens.

Some early coverage, such as Help Net Security’s report, still describes tokens taken from the logs of an internal LiteLLM deployment. That was the initial theory. The later analysis states that the first token was returned by the Agent Platform API.

27 October: a vulnerability rated 10. Security advisory GHSA-w4c7-34wj-hqm8 describes a vulnerability in the LiteLLM proxy running with an Enterprise license. An unauthenticated request to any LLM route on the proxy could cause it to read a local file and send its contents to an address chosen by the attacker. According to the advisory, reading a file such as /proc/self/environ exposes LITELLM_MASTER_KEY and the provider keys, giving the attacker full administrative access to the proxy. In some configurations, the attacker can then execute Python code as root. The CVSS score is 10.0. The fixed versions are 1.103.10, 1.104.8, 1.105.6 and 1.106.5. Open-source deployments running without a license are not affected. The 21 October incident report noted that the private registry was distributing a version that fixed a vulnerability scheduled for disclosure on 27 October. That is the date the security advisory was published.

The common factor: an outbound connection to a destination chosen by the attacker

In both failures, the attacker benefited only if the server running the gateway could open an outbound connection to a destination they chose. For the vulnerability, this is part of the mechanism: the file’s contents are sent to an address specified by the attacker. For the poisoned release, LiteLLM states this explicitly in its cleanup guidance: if 1.107.0 was ever installed on a server with outbound internet access, assume that data exfiltration succeeded.

The coverage adds details about the communication channels. According to Socket’s analysis, cited by Help Net Security, the collected data was encrypted and sent to a “healthcheck” address on the domain litellm[.]dev. A second component repeatedly checked a file on GitHub, from which it was meant to obtain a domain for downloading and executing additional Python code. According to Wiz, as reported in the same coverage, the attacker began testing stolen AWS credentials about four and a half hours after the data left the victims’ systems. They also checked whether those credentials could be used to access models on Amazon Bedrock.

The security advisory itself reaches the same conclusion. LiteLLM recommends several defense-in-depth measures: load the master key from a secrets manager rather than environment variables, configure a separate LITELLM_SALT_KEY, and restrict the proxy’s outbound traffic to known provider endpoints.

There is also a lesson about version pinning. On 21 October, the company recommended pinning 1.106.2. According to the security advisory, the fixed version in that branch is 1.106.5. Anyone running the proxy with an Enterprise license and pinned to 1.106.2 is therefore running a vulnerable version. Pinning protected them from one failure but left them exposed to the other. An outbound allowlist is relevant to both.

Item by item: what an allowlist would block

The lookalike domain. The domain litellm[.]dev contains the name LiteLLM. An allowlist based on the exact hostnames of your configured providers would not have included it, unless someone approved it because it appeared to belong to LiteLLM. The same pattern appeared on GitHub: Berri-AI instead of BerriAI. Approve only a hostname you have checked, not one that merely resembles a familiar name.

The file on GitHub. It is easy to allow access to all of GitHub on the assumption that the gateway needs to get updates from there. That would have left the second component’s instruction channel open. Check whether a running gateway needs access to GitHub at all. Builds and updates can take place elsewhere. If there is a genuine need, approve the exact hostname. Remember that addresses used by a service such as GitHub are shared across several services. According to the coverage, the data-stealing component also works without this file. Both channels therefore need to be blocked, not just one.

webhook.site. In this incident, the attacker used it from GitHub Actions runners, not from a gateway. Even so, its use illustrates the kind of channel attackers favor: a general-purpose service that accepts content from anyone who contacts it. An AI gateway does not need such a service. Paste sites and request capture services belong in the same category. If you run your own runners, remember that they are also servers holding secrets, and they need their own allowlists.

The address chosen by the attacker in the vulnerability. When an allowlist is enforced, connections to addresses not on the list are blocked. The proxy therefore cannot send the contents of the environment file directly to the address chosen by the attacker. However, this restriction does not rule out exfiltration through an approved destination or through DNS. The request itself still reaches the proxy if it is accessible, so reducing inbound exposure is part of the same work.

DNS queries. Data can be hidden in a name sent in a DNS query. A connection allowlist alone does not detect this, so queries also need to be restricted to approved names.

The limits of an allowlist

An allowlist is one layer of protection. To configure the environment properly, you need to know what it does not address:

  • Installation uses an approved channel. Build the image in a separate stage, pin it by digest and run the gateway without access to PyPI.
  • Running code can still read data. The allowlist controls what leaves the server, not what is read from disk or the environment. After exposure, you still need to perform the cleanup and rotate the keys.
  • The restriction needs to cover the entire environment. According to Socket, as reported by Help Net Security, the malicious code was placed in Python startup files. It runs whenever Python starts in the infected environment, even if the application does not import LiteLLM. Run the gateway in its own container or venv, without other Python tools alongside it.
  • Events outside your server need to be addressed outside it. The attack chain began with a key exposed in a public repository and an internal API the attacker could reach. The remedies here are immediate rotation of exposed keys, tokens with limited permissions, and blocking internet access to those interfaces.

Practical steps

  • Check for exposure. Run pip show litellm in every environment, and check whether you ran the image with the infected digest. Also look for litellm-1.107.0.dist-info, cook.pth and sitecustomize.py. According to LiteLLM, a finding confirms exposure, but the absence of findings does not prove the environment is clean. If you were exposed, do not start with pip install. Follow the cleanup guidance, and rotate every credential the server could have accessed: the IAM role, provider keys in the proxy database, the master key and virtual keys, and the DATABASE_URL password.
  • If you have an Enterprise license, update to the fixed version in your branch.
  • Record traffic before writing the allowlist. Observe the destinations the gateway process connects to over a period of time, along with the traffic volume for each destination.
  • Write the allowlist. Include the exact hostnames of the providers you have actually configured, the database (preferably at a private address), the secrets manager if you load the master key from it, and the logging services you deliberately configured. Do not use wildcards.
  • Leave these off the allowlist: GitHub as a whole, PyPI and container registries at runtime, webhook and request capture services, paste sites, and any domain that merely resembles a provider’s name.
  • Enforce restrictions at the network layer and per process, not in the gateway’s settings. On Linux, options include systemd’s IPAddressAllow or an nftables rule based on the service’s cgroup.
  • Restrict DNS queries too, allowing only approved names.
  • Restrict inbound connections. Do not expose the proxy routes to the public internet. Load the master key from a secrets manager and configure a separate LITELLM_SALT_KEY, as recommended in the security advisory.
  • Reduce token permissions on agent servers. In LiteLLM’s case, the agent platform held an organization owner’s GitHub CLI token rather than a token limited to the required repositories and capabilities. Revoke an exposed token immediately.

How nuDefend helps

What is running and where it connects. nudefend workloads shows the AI tools running on the server, whether on the host or inside a container, and the basis for identifying each one. nuDefend records which destinations each AI tool connects to, how much traffic was sent to each destination and when the destination was first detected. When a destination is identified, it is shown by name alongside a description of the activity type. nudefend egress review <workload> shows all destinations for one tool and indicates whether the period was fully monitored. This makes an unusual destination stand out rather than disappear into a list of numbers.

An allowlist built from actual traffic. Initially, nuDefend only records traffic. At the end of the monitoring period, it produces a list based on the traffic observed. On the Outbound screen, you approve the destinations the tool actually needs. For example, an address used by GitHub may also serve other services. Approve such an address only after checking it.

Enforcement below the tool. Once the restriction is enabled, the tool can connect only to destinations you have approved. Any attempt to connect elsewhere is blocked before the traffic leaves the server. The restriction applies only to the selected tool; the rest of the server continues to operate as usual. It can be applied to almost any tool the system identifies. The screen shows whether each tool is restricted. If you do not give final confirmation within ten minutes, the restriction lifts automatically. This lets you enable it on a live gateway, check that the gateway still works, and only then confirm.

The tool’s DNS queries too. A restricted tool can send DNS queries only for names you have approved. Every other query is stopped before it leaves the server. Each blocked query is shown alongside the tool’s name. This is often the first sign that a tool is trying to reach a destination you have not approved.

Threat blocking and inbound connections. A signed threat list, updated every 30 minutes, blocks traffic to and from malicious addresses and domains. On the dashboard, the blocked outbound traffic card turns red as an early warning that something on the server tried to contact a malicious destination. nudefend expose shows every listening service on the server and who can reach it. nuDefend alerts you when a previously internal service becomes accessible from outside. A port that should not be open can be closed with nudefend block add, after a preview showing exactly what will be blocked. Reports remain on the server.

More details are available on the outbound traffic page, the exposure page and in the documentation.

The bottom line

Within one week, LiteLLM published details of two different security failures. Both depended on the same condition: an outbound connection from the gateway to a destination chosen by the attacker. LiteLLM itself now recommends restricting the proxy’s outbound traffic. Version pinning and key rotation address the previous incident. A per-process allowlist addresses the condition itself. Build it from observed traffic, using exact hostnames, without general-purpose relay services or blanket approval for GitHub. Restrict DNS queries too.

nuDefend shows which destinations the AI tools on your server connect to and how much traffic was sent to each one. It lets you restrict a tool to approved destinations. Every other destination is blocked before the traffic leaves the server.

Check outbound traffic.

Ready to protect your servers with nuDefend?