Kubernetes gives a container whole GPUs, even when its work needs a fraction of one [1, 2]. Three NVIDIA methods let several workloads share a GPU: time-slicing, the Multi-Process Service (MPS) and Multi-Instance GPU (MIG) [2]. MIG vs time-slicing is a choice between hardware partitions, each with its own memory, and replicas that take turns on one GPU with no memory or fault isolation between them [5, 17]. MPS sits apart from both: it runs kernels from several processes at the same time, with limits set in software [2, 8].
In a two-month trace from 2020 of one production cluster with over 6,000 GPUs, the median instance used 0.042 GPUs' worth of streaming multiprocessors (SMs), the GPU's compute units [3]. Against a simulated replay of the same trace without sharing, the cluster with sharing needed only half the GPUs on average [3].
Why Kubernetes does not share GPUs by default
Kubernetes offers GPUs to Pods through device plugins. Its documentation says extended resources are only supported as integers and cannot be overcommitted, and that devices cannot be shared between containers [1]. A Pod can ask for a fraction of a CPU and not of a GPU, so an entire physical GPU goes to one container even when it needs only a fraction [2].
| Method | How it shares a GPU | Memory and faults | Main limit |
|---|---|---|---|
| Time-slicing | Replicas take turns on the whole GPU | No memory or fault isolation between replicas [5] | No proportional compute per replica [5] |
| MPS | Kernels from several processes run at once | Own address space per client, limited error containment [13] | One user per server by default [13] |
| MIG | Hardware instances with their own memory paths | Memory and fault isolation per instance [5, 17] | Fixed profiles per GPU model, a GPU reset to turn on [20, 21] |
Time-slicing: replicas that take turns
The NVIDIA device plugin, set up through NVIDIA's GPU Operator, can advertise each GPU as several replicas [5]. With four replicas, a node with two GPUs advertises eight [5]. The GPU gives an equal share of time to all GPU processes, and a Pod that asks for two replicas is not promised twice the compute [5]. The failRequestsGreaterThanOne setting rejects such requests [5]. The device plugin's README recommends turning it on, and it is off by default [6].
What time-slicing gives up is isolation. NVIDIA's documentation says there is no memory or fault isolation between replicas [5], and the README adds that if one workload crashes, they all do [6]. NVIDIA lists the tradeoffs as increased latency, jitter and potential out-of-memory conditions [8]. Amazon's EKS documentation warns that one Pod can exhaust the memory other Pods rely on [9]. Per-container GPU metrics go too: NVIDIA's DCGM-Exporter cannot associate metrics with containers when time-slicing is on [5]. Time-slicing also works on older GPUs that do not support MIG [5].
MPS: kernels from several processes at once
MPS lets kernels from different processes run in parallel on one GPU [8]. Each client has its own GPU address space [13], and NVIDIA describes MPS as useful when a single process does not generate enough work to saturate the GPU [13]. Clients can be held to a device memory limit and an active thread percentage [8, 13]. The thread limit reserves nothing. It only caps what a client can use, and kernels from different clients may run on the same SM [13].
Its isolation is partial. NVIDIA calls MPS error containment limited: a fatal fault is reported to every client on the GPUs where it is contained, without saying which client caused it [13]. Only one user may have an active MPS server on a system, and a multiuser mode drops isolation between users [13]. On GKE, memory bandwidth, encoders and decoders fall outside MPS limits, and Pods need hostIPC: true, which GKE's documentation says introduces security risks [14].
NVIDIA's documentation also covers MPS v3, with named servers and namespaces, while legacy v2 remains the default [12, 15]. Its memory partitioning divides device memory across cgroups and containers and does not work on MIG devices [16]. In the device plugin, MPS splits full GPUs into equal fractions of memory and compute [6].
MIG: hardware partitions from a fixed menu
MIG splits GPUs of the Ampere generation and later into up to seven GPU instances [17, 19]. Each instance has its own paths through the memory system, from crossbar ports and L2 cache banks to memory controllers [17]. NVIDIA says this gives predictable throughput and latency even when other tasks thrash their own caches or saturate their DRAM interfaces [17].
The sizes come from a fixed menu for each GPU model. On an H100 80GB, the profiles run from seven 1g.10gb instances through 2g.20gb and 3g.40gb to one 7g.80gb [20]. Changing MIG mode requires a GPU reset [21, 24], and NVIDIA's MIG Manager terminates the GPU Pods on a node before it enables MIG mode [22]. In Kubernetes, the single strategy shows every MIG device as nvidia.com/gpu, with one profile across the node, and the mixed strategy names each profile as its own resource, such as nvidia.com/mig-1g.10gb [22].
MIG vs time-slicing vs MPS in measurements
A 2026 study on an A30 found that MPS improved performance by up to 30% in favorable pairings and worsened it by around 30% under memory contention [25]. MIG gave more consistent gains at a higher overhead, and memory-intensive pairs finished almost twice as fast under MIG as under MPS [25]. In the memory-intensive pairs with the worst slowdowns, those with the Sort test, average memory utilization reached roughly 60% under MPS, against barely over 20% under time-slicing [25].
A second 2026 study, on an A100 and an edge GPU with image classification models, found that MIG gave more robust temporal isolation than MPS and Green Contexts, though timeouts still appeared under MIG with two processes [26]. A study of model training on an A100 found that collocating jobs could give up to four times the throughput, and its authors recommend MPS for one user's training jobs [27].
MIG's isolation has edges. Contention persists through shared components such as PCIe bandwidth, and MIG's few valid configurations and rigid placement fragment a GPU as jobs arrive and finish [28]. Amazon notes that compute is not strictly proportional to instance size [24]. An October 2026 preprint reported covert channels across MIG instances on one platform, the Grace Hopper GH200, at roughly 8,500 bits per second [29].
Setting it up on Kubernetes
The device plugin applies time-slicing or MPS to every GPU on a node, never to one GPU alone, and the two are mutually exclusive [6]. Shared GPUs carry a product label ending in -SHARED, so a node selector can ask for shared or exclusive GPUs [5]. Installing the operator, drivers and node labels belongs to the wider subject of GPU scheduling on Kubernetes. Batch training on GPU clusters, and how one cluster runs both schedulers, is covered in Slurm or Kubernetes for GPU clusters: how to choose.
Dynamic Resource Allocation (DRA), which the Kubernetes documentation lists as stable since v1.35, lets Pods share a device by referencing the same resource claim [30]. The CNCF's conformance checklist for Kubernetes v1.37 lists GPU sharing at level SHOULD: for accelerators that support static sharing, at least one sharing strategy, hardware partitions exposed as schedulable resources where the accelerator supports partitioning, and oversubscription where it supports software sharing such as time-slicing [31]. Inference services on shared GPUs still need the routing, scaling and rollbacks set out in Deploying LLMs on Kubernetes: serving, scaling and rollbacks.
Across tenants: trust decides the method
GKE's documentation says only workloads in the same trust boundary should use time-sharing, GKE's name for time-slicing [10]. MPS serves one user per server, and its multiuser mode drops isolation between users [13]. NVIDIA presents MIG for cloud providers with several tenants, saying one client cannot affect the work or scheduling of others [17]. The authors of the 2026 preprint write that their results call MIG's isolation guarantees into question [29]. With the NVIDIA DRA driver, sharing cannot cross namespaces, because the claims are namespace-scoped [9].
Read together, these sources fit time-slicing and MPS to workloads that trust each other, and MIG to tenants who must stay apart [10, 13, 17].
Sizing a slice from memory first
Memory is the first number to check. A MIG profile fixes the memory of each instance [20]. Time-slicing enforces no memory limit, and GKE advises setting GPU memory limits in the workloads themselves [10]. For a hypothetical workload that needs 15 GB, a 1g.10gb instance on an H100 80GB is too small, and a 1g.20gb or 2g.20gb fits, up to four or three per GPU (computed here from [20]). The weights, key-value cache and engine reserve that set that memory figure for an LLM are worked out in How much GPU memory LLM inference needs. Whether a shared GPU pays for itself then turns on how busy it stays, the subject of Self-hosted LLM vs API cost: a break-even method.
Rillor designs and builds Kubernetes platforms for superintelligence (SI) work on GPUs, on servers the client controls, where GPUs are offered to the scheduler through the device plugin and a workload gets only the GPUs it requests.
Read the full report
The full report adds the MIG profile menus of five GPU models, MPS v3, the three studies in detail, sharing through resource claims, and all 31 sources.