A progressive, vendor-neutral series on building and operating AI/ML platforms — from MLOps foundations through Kubernetes GPU resource sharing (MIG/MPS, HAMi, Volcano, Kueue, DRA).
33 articles in this category
How to share a Kubernetes-based AI platform across teams without letting one tenant starve the others — covering isolation models, quota mechanics, priority classes, and network boundaries.
A practical breakdown of the six roles on an AI platform team: job-to-be-done, skill bar, day-one tools, and the anti-patterns that surface when a role is mis-hired.
Centralised platform team, embedded MLOps, federated ownership, centre-of-excellence: the four org patterns for ML, when each fits, and the failure mode that ends each one.
Model cards, lineage chains, and audit trails mapped to the four questions every regulator asks: what is it, where did it come from, who approved it, and is what you're serving what you signed?
How SLSA levels, Cosign keyless signing, and CycloneDX ML-BOM extend container supply-chain security to model artifacts — with an end-to-end example from registry promotion to admission gate.
Three-class triage for ML incidents, runbook stubs for drift alerts, latency spikes, and prompt injection, plus the blameless post-mortem template that actually captures lessons.
How to extend SLSA provenance, Cosign signing, SBOMs, and admission-time verification from container images to model artefacts — covering the full chain from training run to serving pod.
DRA reached GA in Kubernetes v1.34. Here is why it exists, what the resource.k8s.io API looks like, and the honest adoption caveats for today's clusters.
Apply the FinOps Inform→Optimise→Operate loop to GPU-hours and tokens: a four-stage maturity ladder, a Kubernetes label scheme that survives an audit, and worked unit-economics arithmetic for training and inference.
How distributed ML training maps onto Kubernetes: PyTorchJob, MPIJob, and RayJob; why gang scheduling is a correctness requirement; and how to pick your checkpoint cadence.
Why BLEU and ROUGE fail LLM systems, how LLM-as-judge works, and how to build a deterministic CI gate from probabilistic scores.
A vendor-neutral mapping of EU AI Act obligations and the NIST AI RMF to concrete platform controls, with a risk-tier decision tree and the four operational practices every AI platform team needs.
HAMi virtualises GPU memory and compute at the CUDA-call layer — hard memory caps, soft compute throttling, no MIG-capable silicon required.
GPU idle time is expensive. This article explains the utilisation economics that make sharing necessary, then maps four sharing mechanisms across the axes that matter: memory isolation, compute isolation, fault isolation, and latency jitter.
The four deployment contexts — pure-cloud, on-prem, hybrid, and regulated air-gapped — defined across five axes, with a warning about recommendations that silently assume pure-cloud.
Five levels from ad-hoc to self-improving, the evidence pattern at each, and the single highest-leverage move that advances you to the next.
Four serving runtime families, the predictor/transformer/explainer pipeline pattern, routing-layer trade-offs, and the autoscaling signals that hold up under real inference traffic.
Six open problems that no production AI platform has fully solved in 2026: eval confidence intervals, agent runtime standards, GPU-side provenance, regulator-readable lineage, the DRA migration, and multi-cluster quota federation.
The AI gateway as a platform primitive: key management, spend enforcement, semantic caching, routing/failover, guardrails, and OTel-GenAI observability — with a decision tree and four-gateway comparison.
Why a model registry is non-negotiable at platform scale, the three registry patterns, four lifecycle states with explicit gates, and curation policy as code.
What classical MLOps disciplines carry into LLM systems, what is genuinely new, and the three wrongly-assumed-new capabilities that waste re-investment.
Prompt templates and tool definitions determine LLM system behaviour as surely as model weights. Here is the versioning, registry, and rollback architecture that keeps them under control.
How to select MIG profiles for A100, H100, and B200 GPUs, avoid the fragmentation trap that catches every team, and observe per-partition metrics with DCGM.
How to wire metrics, traces, and SLOs across LLM calls, tool invocations, and agent turns using the OpenTelemetry GenAI semantic conventions.
How Kueue, Volcano, and the NVIDIA GPU Operator compose into a three-layer stack — and where each layer fails when something goes wrong.
How to decompose RAG failures into three distinct modes, measure faithfulness and relevance independently, and grade agent trajectories with OTel GenAI conventions.
Batch and online inference have opposite optimisation functions. This article covers resource shape, latency metrics, autoscaling signals, and how to size GPU capacity for a target p95.
How platform teams define, template, and evolve the three canonical ML golden paths — training pipeline, model serving, and GenAI feature — and why the deprecation contract matters as much as the path itself.
MLOps defined — not as a tools list but as a discipline. Four reference framings, three points of real disagreement, and a vocabulary the rest of this series builds on.
A defensible decision tree routing you to time-slicing, MPS, MIG, HAMi, or no sharing — based on GPU SKU, isolation requirements, and workload mix.
Training, fine-tuning, batch inference, online inference, and agent — a decision framework for classifying any AI/ML workload by resource shape, latency budget, lifecycle, failure mode, and scheduling fit.
Four adaptation modes — full fine-tune, LoRA, QLoRA, RLHF/DPO — mapped to GPU memory, data shape, wall-clock, and the RAG-vs-fine-tune decision rule every ML team needs.
Walk the eight-stage ML lifecycle from problem framing to model retirement, and learn what changes at each stage when you move from a notebook to a production system.