Lyrie
Supply-Chain
0 sources verified·8 min read
By Lyrie Threat Intelligence·5/12/2026

The Poisoned Gateway: LiteLLM PyPI Supply Chain Attack Weaponizes AI Model Routing

TL;DR

On March 24, 2026, TeamPCP compromised the LiteLLM Python package on PyPI (versions 1.82.7 and 1.82.8) via poisoned Trivy GitHub Action credentials, injecting a three-stage backdoor with credential harvesting, persistent systemd hooks, and Kubernetes lateral movement. The package sees 3.4 million downloads daily. The malicious versions escaped detection for ~3 hours and were found by developers testing AI coding agents experiencing RAM exhaustion from fork-bomb persistence hooks.

What Happened

On March 24, 2026, at 10:39 UTC, a malicious version of the LiteLLM Python package (1.82.7) appeared on PyPI, followed by an escalated variant (1.82.8) just 13 minutes later. Both versions were uploaded using legitimate PyPI credentials belonging to the LiteLLM maintainer, obtained through a prior supply chain compromise.

The attack was discovered by Callum McMahon at FutureSearch AI when his development machine became unresponsive during RAM testing of a Cursor AI coding agent. He traced the exhaustion to a newly installed LiteLLM dependency and found a suspicious 34,628-byte file named litellm_init.pth in Python's site-packages directory — a double base64-encoded payload that executes on every Python interpreter startup.

The malicious packages remained available on PyPI for approximately three hours before being quarantined. During that window, they were installed by multiple critical AI/ML projects including DSPy, MLflow, OpenHands, CrewAI, Arize Phoenix, and others.

Technical Details

The Attack Chain: Five Days of Escalation

March 19: TeamPCP compromised the trivy-action GitHub Action repository, rewriting Git tags for v0.69.4 to point to a malicious release. This release carried credential harvesting code targeting the GITHUB_TOKEN and PYPI_PUBLISH token.

March 23: The same infrastructure was reused in a separate attack on Checkmarx KICS (Keep Infrastructure as Code Secure), registering the lookalike domain checkmarx.zone as a command-and-control endpoint.

March 24 (10:39 UTC): LiteLLM's CI/CD pipeline pulled the compromised Trivy GitHub Action as part of its build process. The malicious Trivy exfiltrated the PYPI_PUBLISH token from the GitHub Actions runner environment. With this credential, TeamPCP published litellm 1.82.7 containing a base64-encoded payload embedded directly in litellm/proxy/proxy_server.py.

March 24 (10:52 UTC): Recognizing the detection risk from the source injection approach, TeamPCP published 1.82.8 with an escalated delivery mechanism: a .pth startup hook file that fires on every Python process start, including when pip or IDEs initialize Python interpreters, with zero imports required.

Two Delivery Mechanisms, Same Result

Version 1.82.7 (Source Injection): The payload was base64-encoded and embedded directly in litellm/proxy/proxy_server.py. It executed when anything imported litellm.proxy, the standard import path for LiteLLM's proxy server mode.

Version 1.82.8 (.pth Hook — the escalation): This version added litellm_init.pth to the Python site-packages directory. The .pth mechanism is a CPython feature designed for package initialization; it executes on every Python interpreter startup with no import required — including pip operations, IDE language servers, and python -c one-liners. The file passed pip's hash verification because it was correctly declared in the wheel's RECORD metadata; the package carried no hash mismatches or suspicious artifacts because it was published with legitimate credentials.

Three-Stage Payload: Harvest, Exfiltrate, Persist

Stage 1: Credential & Secrets Collection

The payload performed systematic sweeps of every major credential plane in cloud-native environments:

  • GitHub Actions: Direct reads of GITHUB_REPOSITORY, ACTIONS_ID_TOKEN_REQUEST_TOKEN, and enumeration of GitHub Actions secrets via the REST API
  • AWS: Environment variables, IMDSv2 metadata, ECS task metadata, Secrets Manager and SSM Parameter Store enumeration across all regions
  • HashiCorp Vault: VAULT_TOKEN, VAULT_ADDR, and direct API calls to the Kubernetes-internal endpoint (vault.svc.cluster.local:8200)
  • Kubernetes: Service account tokens under /var/run/secrets/kubernetes.io/serviceaccount/
  • Workstations: ~/.npmrc, ~/.git-credentials, SSH private keys, GCP metadata files, ~100+ cloud credential paths on Linux and macOS
  • Cryptocurrency: Bitcoin, Ethereum, Solana, Cardano, Monero wallet files and seed phrases
  • Development Tools: Slack/Discord webhook tokens, Docker registry credentials, CI/CD configs (Jenkins, Travis CI, Terraform)

Stage 2: Encrypted Exfiltration

Collected credentials were bundled into tpcp.tar.gz, encrypted with AES-256-CBC (using a session-specific 32-byte key derived via PBKDF2), and the session key was wrapped with a hardcoded 4096-bit RSA public key (OAEP padding). The encrypted bundle was POST'd to https://models.litellm.cloud/ — a domain registered on March 23, one day before the attack, distinct from the legitimate litellm.ai domain.

Temporary exfiltration files were created in system temp directories: session.key, payload.enc, session.key.enc, tpcp.tar.gz.

Stage 3: Persistence & Lateral Movement

The payload wrote a backdoor Python script to ~/.config/sysmon/sysmon.py and registered a systemd user service at ~/.config/systemd/user/sysmon.service with the benign description "System Telemetry Service." This service was enabled and started immediately.

The persistence script polled https://checkmarx.zone/raw every 5 minutes for a URL, downloaded files to /tmp/pglog, made them executable, and ran them as background processes. At the time of analysis, the endpoint returned YouTube URLs; the operator can switch to delivering live payloads at will.

Kubernetes Lateral Movement: If the script detected a Kubernetes service account token at the standard mount path, it read all secrets across every namespace and attempted to deploy privileged pods to every node in kube-system, mounting the host filesystem and installing the backdoor on the underlying infrastructure.

The Fork Bomb Discovery

The .pth mechanism fires on every Python process start. Because the payload spawned a new Python subprocess, and that subprocess also triggered .pth execution, the result was an unintended recursive fork bomb that exhausted RAM on affected developer machines. This unintended side effect is what allowed Callum McMahon to discover the compromise during his testing.

Attribution: TeamPCP's Operational Continuity

The RSA public key used for payload encryption in LiteLLM was identical to the keys found in the prior Trivy and KICS attacks, providing strong technical attribution to TeamPCP (also tracked as PCPcat, Persy_PCP, ShellForce, and DeadCatx3). The consistent use of the tpcp.tar.gz bundle naming convention and the same registrar/hosting provider (Spaceship Inc. / DEMENIN B.V.) across all three operations demonstrates coordinated campaign execution.

Wiz researchers confirmed TeamPCP's operational model also includes deployment of CanisterWorm, which uses the Internet Computer Protocol (ICP) as a C2 channel — the first observed use of ICP for supply chain attack command-and-control. The group also deploys hackerbot-claw, an AI agent (openclaw) for automated attack targeting.

Issue Suppression

When the community began reporting the compromise in GitHub issue #24512, the attackers posted 88 bot comments from 73 unique accounts in a 102-second window, using previously compromised developer accounts rather than purpose-created profiles. Using the hijacked krrishdholakia maintainer account, they closed the issue as "not planned" and made commits with "teampcp update" messages. The community opened a parallel tracking issue (#24518) and continued disclosure on Hacker News, where the thread reached 324 points.

Lyrie Assessment

This incident exemplifies the elevation of supply chain attacks into AI infrastructure layers. LiteLLM is increasingly deployed as a centralized gateway storing API credentials for multiple model providers — a single compromised endpoint can harvest credentials across an organization's entire LLM access layer.

The three-stage payload design shows sophisticated operational maturity: credential harvesting is separated from exfiltration (allowing selective monitoring), which is separated from persistence (enabling long-term foothold maintenance). The Kubernetes lateral movement adds a critical-infrastructure angle; compromised DevOps pipelines using LiteLLM can serve as the bridgehead for cluster-wide compromise.

The three-chain attack pattern (Pwn Request → compromised Trivy → PyPI credentials) is now the expected TTPs for TeamPCP and similar supply chain actors:

1. Compromise a widely-used security tool via GitHub Actions misconfiguration

2. Use that tool's access to exfiltrate upstream credentials

3. Weaponize the upstream platform (registry, artifact repository, package manager)

For CISOs: LiteLLM and similar centralized AI model routers are becoming high-value targets. They consolidate multiple model provider credentials in a single place and require broad read access to system configuration and environment variables by design. Organizations deploying centralized LLM gateways should implement:

  • Credential rotation on a monthly cycle, not on-demand only
  • Separate, zero-trust service accounts for LLM routing that are never stored on developer workstations
  • Supply chain scanning of every transitive dependency in AI tooling stacks
  • Immutable audit logging of all credential access from LLM gateway processes

Recommended Actions

1. Immediate: Check if any development environment installed litellm versions 1.82.7 or 1.82.8 between March 24, 10:39-13:48 UTC. The .pth payload executes when Python starts, including during pip install itself; treat any affected system as compromised regardless of whether application code ran.

2. Credential Rotation (execute in this order):

- SSH keys: revoke from authorized_keys, GitHub, GitLab, all platforms

- AWS access keys: rotate immediately via IAM console

- GCP/Azure: regenerate service account keys

- Docker registry credentials: revoke from registry and refresh ~/.docker/config.json

- Kubernetes: rotate service account tokens and audit all RBAC changes

- PyPI, npm, GitHub: rotate all tokens with lifetime >7 days

- Database passwords and API keys

3. Persistence Artifact Cleanup (execute in this order):

- rm -rf ~/.config/sysmon/

- systemctl --user disable sysmon

- Check for malicious .pth files in Python's site-packages

- Audit ~/.config/systemd/user/ for unexpected services

4. Kubernetes Audit:

- Query kube-system for node-setup-* pods

- Audit all secrets across all namespaces for exfiltration timestamps

- Review all RBAC changes in audit logs

- Rotate service account tokens cluster-wide

5. Upgrade Path:

- Pin litellm to ≤1.82.6 in all dependency files

- Do not upgrade in-place; install clean version on fresh environment

- Test on isolated network before redeploying to production

6. Detection & Verification:

- Run pip list | grep litellm and verify version ≤1.82.6

- Hash check: sha256sum $(python3 -c "import litellm; print(litellm.__file__)") against known-good manifests

- Check for litellm_init.pth files in all Python site-packages directories

- Query Snyk's SNYK-PYTHON-LITELLM-15762713 record for latest detection rules

Sources

1. Snyk: How a Poisoned Security Scanner Became the Key to Backdooring LiteLLM

2. FutureSearch AI: LiteLLM PyPI Supply Chain Attack Analysis

3. Endor Labs: TeamPCP's LiteLLM Campaign Phase 09

4. Snyk Security Database: SNYK-PYTHON-LITELLM-15762713

5. Wiz Threat Center: TeamPCP Threat Actor Profile

6. Rami McCarthy: Technical Attribution — Trivy, KICS, LiteLLM Infrastructure

7. GitHub Issue #24512: Initial Disclosure and Community Response

8. GitHub Issue #24518: Parallel Tracking & Remediation Guidance


Lyrie.ai Cyber Research Division

Lyrie Verdict

Lyrie's autonomous defense layer flags this class of exposure the moment it surfaces — no signature update required.