Self-hosted LLM security covers every part of the stack an organization takes on when it runs a model itself: the inference server, the model files, the cache that speeds up repeated prompts, the document index behind retrieval and the GPU container layer underneath. Each part has published failures, and the sources that document them name controls.
This explainer goes through six documented failure modes, maps each to the OWASP Top 10 for LLM Applications 2025 and NIST AI 100-2, and lists the controls the sources give. Tool permissions for agents are the subject of the MCP access control article.
What changes in self-hosted LLM security
NIST's 2025 taxonomy of attacks on machine learning systems advises addressing security "by considering the full attack surface, including data and model supply chains, software, and network and storage systems" [26]. A self-hosted deployment brings that whole surface inside the organization. The explainer on private LLM deployment sets out which party holds the hardware, the weights and the logs in each arrangement, and the break-even method for self-hosted LLM cost prices the move.
Several of the tools ship with defaults suited to one trusted machine or network. Ollama binds to 127.0.0.1 by default [4], and Wiz noted in 2024 that it "does not support authentication out-of-the-box" [5]. vLLM states that traffic between the nodes of a multi-node deployment is "insecure by default" [7].
Exposed inference endpoints
Inference servers open to the internet are common enough to count. Cisco Talos searched Shodan for Ollama servers and on 1 September 2025 reported 1,139 exposed instances, 214 of them responding with live models [1]. A test prompt that came back answered showed the server was "accepting and executing prompts without requiring authentication" [1]. Censys reported on 24 September 2025 that it found about 10,600 exposed Ollama instances across 1,229 autonomous systems, with about 1,500 answering at least one test prompt and over 25% on a port other than the default [2]. The two scans used different methods.
Exposure is a deployment choice. Ollama's documentation shows how to change the bind address so the server listens on every interface [4]. Wiz wrote in 2024 that Docker installations of Ollama listened on all interfaces by default and advised a reverse proxy to enforce authentication [5]. vLLM's documentation warns that its API key flag protects only endpoints under four path prefixes, while others on the same server, including ones that pause the engine or update its weights, have no authentication [7].
Open endpoints get found. Pillar Security's honeypots captured 35,000 attack sessions against "exposed AI infrastructure" between December 2025 and January 2026, an average of 972 a day, with targets including Ollama on port 11434 without authentication and chat-completions APIs on port 8000 [3]. Pillar reports that exploitation attempts "typically begin within hours" of an endpoint appearing in Shodan or Censys results [3].
Flaws in serving software
Inference servers are software with their own CVEs. Probllama, CVE-2024-37032, let a crafted model manifest overwrite arbitrary files on an Ollama server through path traversal, which Wiz showed could lead to remote code execution [5, 6]. The NVD record carries a CVSS 3.1 score of 8.8 from CISA's data publisher, and NVD has not scored it [6]. Wiz called the issue extremely severe in Docker installations, where the server ran with root privileges [5]. A patched release followed three days after the report, and Wiz asked users to upgrade to version 0.1.34 or newer [5].
A second pattern spread by copying. Oligo Security found that several inference frameworks, vLLM and SGLang among them, carried nearly identical code that deserialized Python pickle data over unauthenticated ZeroMQ sockets, and that one SGLang file began "Adapted from vLLM" [9]. A separate vLLM flaw of the same kind, CVE-2025-32444, rated 9.8 by NVD, affected deployments using its mooncake integration from version 0.6.5 until 0.8.5 [8]. vLLM asks operators to place multi-node deployments on an isolated network and to block all incoming connections except to the API server's port [7].
Model files that run code
A model file can be a program. Hugging Face's documentation warns of "dangerous arbitrary code execution attacks" when a pickle file is loaded, and calls its own scanner "not 100% foolproof" [10]. JFrog reported around 100 models with real harmful payloads on the Hugging Face Hub in February 2024, one of which opened a reverse shell when loaded [11]. In February 2025 ReversingLabs described two malicious models the Hub's scanner did not flag, packed with 7z instead of PyTorch's default ZIP format and carrying a broken pickle stream [12].
PyTorch has loaded with weights_only=True by default since version 2.6, which "narrows the surface of remote code execution attacks" [13]. The restricted loader had a flaw of its own: CVE-2025-32434 allowed remote command execution through weights_only=True in version 2.5.1 and earlier [14]. Hugging Face points to safetensors as the alternative format, and a 2023 audit by Trail of Bits found no critical flaw leading to arbitrary code execution [10, 15]. Custom model code loaded with trust_remote_code is a separate path, and Hugging Face advises loading it from a specific revision [16].
Shared KV caches between tenants
Serving engines keep a key-value (KV) cache for the tokens they have processed, and engines such as vLLM and SGLang can share it across requests with an identical prompt prefix [7, 17]. At NDSS 2025, Wu and colleagues showed that this sharing in multi-tenant servers can leak other users' prompts. Their attack, run as a standard user without special privileges, reached an average success rate of 99% in fully or partially reversing the prompt input on a 13-billion-parameter open-weight model [17]. A separate ICML 2025 study detected global cache sharing across users in seven API providers [18].
vLLM fixed one timing leak, CVE-2025-46570, in version 0.9.0 [19]. Its documentation describes a cache salt that limits prefix sharing to requests carrying the same salt. The salt is "opt-in and not passed by default", and vLLM asks multi-tenant deployments to set one on every request, scoped to the tenant boundary, at some cost in cache efficiency [7].
Poisoned documents and vector indexes
Retrieval opens a channel for instructions. Greshake and colleagues showed in 2023 that text planted in data likely to be retrieved can steer an LLM-integrated application without any direct access to it [20]. PoisonedRAG reached a 90% attack success rate by adding five malicious texts per target question to a knowledge database holding millions of texts, and found the defenses it tested insufficient [21]. The index is sensitive in its own right: one EMNLP 2023 method recovered 92% of 32-token inputs exactly from their embeddings [22]. How to test an agent against planted instructions is covered in prompt injection testing.
The GPU container layer
The NVIDIA Container Toolkit sits between GPU containers and the host, and two flaws NVIDIA rated critical let a container reach that host. CVE-2024-0132, published 26 September 2024, let a crafted image gain access to the host file system in the default configuration, and did not affect use with the Container Device Interface (CDI) [27]. Wiz wrote that in a shared cluster, an attacker allowed to deploy a container could escape it and reach other applications' data and secrets [28]. CVE-2025-23266, published 17 July 2025 and named NVIDIAScape by Wiz, sat in the hooks that set up a container and gave a malicious container full root access to the host [29, 31]. Wiz lists CDI mode as affected [31], so switching to CDI did not close every path.
The fixes came as releases: toolkit 1.16.2 and GPU Operator 24.6.2 for the first flaw, and toolkit 1.17.8, device plugin 0.17.3 and GPU Operator 25.3.2 for the second [28, 30, 33]. Both flaws arrive inside the container image, so Wiz advises prioritizing hosts that run untrusted or public images, and says containers should not be the only isolation barrier [28, 31].
Controls for each failure mode
The OWASP entries in the table are this article's mapping onto the 2025 list [23].
| Failure mode | OWASP 2025 entry (mapped here) | Controls the sources give | Sources |
|---|---|---|---|
| Exposed endpoints | LLM10 Unbounded Consumption | Authentication on every request, a reverse proxy that allows listed endpoints only, rate limits | [1, 3, 7] |
| Serving software flaws | LLM03 Supply Chain | Patched versions, isolated networks between nodes, no pickle over untrusted sockets | [7, 9, 24] |
| Model files that run code | LLM03 Supply Chain | Safetensors or weights-only loading on a patched PyTorch, verified sources, pinned revisions | [10, 13, 14, 15, 16, 24] |
| Shared KV caches | LLM02 Sensitive Information Disclosure | A secret cache salt per tenant boundary | [7] |
| Poisoned documents | LLM08 Vector and Embedding Weaknesses | Permission-aware vector stores, partitioned datasets, trusted sources only | [25] |
| GPU container escapes | LLM03 Supply Chain | Patched toolkit releases, a stronger isolation barrier such as virtualization | [30, 31, 35] |
This article places the cache leak under NIST AI 100-2's privacy compromise, poisoned documents under indirect prompt injection and malicious model files under supply chain attacks [26].
In a private superintelligence (SI) cloud that Rillor deploys, each tenant runs in its own isolated space with its own network rules and credentials, people get role-based access, agents and tools get scoped credentials, and sign-ins, changes and agent tool calls are logged with time, identity and result. Rillor's agentic security work tests SI agents against attack, then limits what they can reach, and it tests only systems the client owns or is authorized to test, within a written scope.
Read the full report
The full report adds the scan methods and their limits, the conditions each attack needs, the toolkit's CVE history release by release, the full OWASP and NIST mapping and all 36 sources.