Most AI systems that struggle in production are not struggling because the model is
wrong. They struggle because the infrastructure underneath was never designed for how
the workload actually behaves.
The symptoms are consistent across every engagement I have taken. Expensive GPUs sit
idle while jobs queue behind them. One team's training run starves another's. The cloud
bill grows in a way nobody can attribute to a decision. And the whole arrangement is
held together by a few people who know which thing to restart at 3am.
The gap between a model that works and a platform that runs it reliably and affordably
is where I have spent my career. Below is how I got here, what I have measured, and how
I approach the work.
02 Where it started
MSCS thesis, NUST Islamabad, 2021
In 2021 I was trying to explain why GPU clusters cost so much more than their measured
utilization seemed to justify. The answer turned out to be structural rather than
operational. Kubernetes will only schedule a GPU workload onto a node that has a GPU
physically attached to it. If a workload needs a GPU for thirty seconds an hour, that
card still has to live in that machine, doing nothing for the other fifty-nine minutes.
Hardware cost scales with the number of machines that might need a GPU, not
with the work actually being done.
My thesis removed that binding. Every GPU in the data centre goes into a single pool. A
workload running on a node with no GPU attached requests one at runtime, uses it, and
releases it back, so the same physical card serves workloads on different machines at
different times. I built it end to end rather than simulating it: VMware ESXi and
vCenter as a private cloud over bare metal, Bitfusion as the pooling layer, a Kubernetes
cluster on top, and Rancher for provisioning and fleet management.
The only question that mattered was what it cost in performance. Benchmarked against a
directly attached GPU on identical CUDA 11 workloads:
Pooled GPU versus directly attached GPU
Benchmark
Operations
Attached
Pooled
Overhead
matrixMul
131,072,000
0.130 ms
0.132 ms
1.5%
matrixMulCUBLAS
196,608,000
0.056 ms
0.059 ms
5.4%
The overhead is network transport of CUDA calls and data pointers, not slower GPU
execution, and improving the network between hosts is far cheaper than buying a card for
every machine. The conclusion held: you can drop per node GPU allocation entirely and
pay a few percent of latency for it.
This was 2021, before GPU disaggregation was a commercial category. The thesis closed by
proposing the same pooling across clusters, and then across clouds. That is close to a
description of what I have built in production every year since.
One idea generalises across everything I have done since: when infrastructure is
expensive, the default is to give every consumer their own and absorb the waste. Almost
always, the better answer is to pool it and share it safely. That progression has three
stages, and I have worked through them in order.
Stage one Pool the hardware
GPU orchestration and cost control across AWS, Azure, and bare metal: scheduling,
workload distribution, and getting utilization up before buying anything. On a real time
gaming platform this meant cutting AWS spend by more than 60%, through better model
selection and inference harness work on Bedrock plus disciplined rightsizing of compute
and storage. The cheapest GPU is the one you did not need to buy.
Sometimes pooling means building the substrate yourself. I automate full OpenStack
private clouds on bare metal with Ceph behind them, so a rack of machines becomes pooled
compute, storage and network that teams can draw from on demand. I build these with
Ansible roles and playbooks rather than a containerised control plane, which keeps the
failure modes inspectable and the whole deployment re-runnable, so expanding a cluster or
rebuilding it is routine work rather than a project.
Stage two Share the platform
Pooled hardware is only useful if several teams can use it without interfering with each
other. I build multi-tenant Kubernetes platforms: Kamaji based control planes so each
tenant gets real isolation rather than a shared namespace, Gateway API for routing across
tenant workloads, and Pulumi modules in Go that provision a tenant's Kubernetes, GPU
pools, and MLOps stack as typed infrastructure code, with automated teardown so a
finished tenant stops costing money. Where data cannot leave the building, I deliver the
same thing as vanilla Kubernetes on customer owned hardware.
Stage three Automate the operation
Once a platform is pooled and multi-tenant, the largest remaining cost is human
attention. This is where agentic systems earn their place, and where most of them fail,
because a demo that reasons well is not the same as something safe to leave running. I
have shipped a LangGraph based SRE harness that triages incidents, classifies alerts
dynamically, and gates remediation behind human approval, a Marvin based harness for
content generation, and a training pipeline harness with subagent failure recovery,
dynamic branching for hyperparameter sweeps, and per model skill loaders. Every one of
them was built with a token budget and a stopping condition, because an unbounded agent
loop is a billing incident waiting to happen.
04 Evidence
Result
Measure
Where
AWS spend reduced
60%+
Real time gaming platform
Cluster provisioning, production ready with GPU
< 10 min
Kubernetes automation platform
Overhead of pooled GPU versus dedicated
1.5%
MSCS thesis, measured
Microservices with build and release automated, air gapped
40+
Enterprise environments
Distributed training alerting owned in production
3.5 yrs
Multiple client engagements
Production uptime sustained
99.9%
Across client platforms
Numbers without a mechanism are decoration, so each of these is explained above or in
the track record below. Where a figure needs a baseline to be meaningful, ask me and I
will give you the baseline.
05 How I work
Blast radius before convenience
Read paths stay open, mutating paths get gated. Most production incidents I have been
called into were caused by a change that was easy to make and hard to reverse.
Measure before optimising
A baseline first, then the change, then the same measurement again. Without the first
step, a cost or performance claim is a guess wearing a percentage sign.
Boring where it counts
Novel technology belongs where it creates advantage, not in the load bearing parts.
I would rather run a dull control plane and spend the innovation budget on the
workload.
Every agent gets a budget and a stopping condition
Token budget, step cap, and a human approval gate on anything that mutates
infrastructure. Autonomy is earned per capability, not granted once.
Documentation that survives being checked
If the runbook and the cluster disagree, the runbook is a liability. I would rather
write less and have it be true.
06 Track record
These are contract and consulting engagements, delivered through
Eprecisio Technologies, the infrastructure and AI
consultancy I founded. Dates overlap because they ran in parallel rather than in
sequence.
Sep 2024 to present
Stack8s Senior DevOps Engineer, contract
Kubernetes automation for AI and ML teams. Took production ready cluster provisioning
with GPU compute to under ten minutes across AWS, Azure and on premise, with Kubeflow
ready to use on arrival. Built the Kamaji based multi-tenant layer with Gateway API
routing, run Rancher fleet management across multi-cloud and on premise estates, and
delivered vanilla Kubernetes onto customer owned hardware for regulated environments
where data sovereignty ruled out the cloud. Also automated a full OpenStack deployment
on bare metal with Ceph as the storage backend, using custom Ansible roles, playbooks
and group_vars rather than a containerised control plane: Nova, Neutron, Cinder,
Keystone, Glance, Horizon and Ceilometer, with OVS, VXLAN and DVR networking, built
re-runnable so redeployment and cluster expansion are routine.
Apr 2025 to Jul 2026
Hyve Labs Senior DevOps Engineer, contract
Owned production for a real time gaming platform: event fan out, elastic multi region
autoscaling, low latency multiplayer backends, and the release lifecycle. Cut AWS spend
by more than 60%. Rebuilt the data and ML stack on Bedrock, Airflow, Redshift and Glue
around Kafka driven pub/sub feeding analytics, model workflows, and downstream game
systems. Built the secure production layer, layered IAM and RBAC, secrets, image
hardening and network policy, ran regression, load and stress testing plus periodic
penetration testing, and owned PagerDuty escalation with P0 to P2 SLA enforcement.
Designed Ray API and Ray head cluster architectures for distributed training and
inference, on bare metal Kubernetes with MetalLB and on managed cloud. Built the
Pulumi and Go modules behind a multi-tenant AI platform, with per tenant provisioning
and automated teardown. Built the training pipeline agentic harness. Owned production
alerting for distributed training across three and a half years of client engagements:
GPU utilization, cluster health, cost anomalies, and silent failure detection.
Apr 2020 to Sep 2024
Aceso Analytics DevOps Engineer and Team Lead
Led the engineering team on a healthcare IoT platform serving LoRaWAN device fleets
across elderly care facilities, architected on Kubernetes with HIPAA aligned
operational practice. Built the monitoring, alerting and edge inference for real time
device fleets, and mentored the junior engineers on the team.
Automated build, packaging and release for more than 40 microservices inside air gapped
enterprise environments, and migrated legacy infrastructure onto on premise Kubernetes
with Ansible playbooks and Jenkins shared libraries. Ran parallel engagements for US
and international clients across AWS, Azure and GCP.
07 Credentials and tooling
Certifications
Certified Kubernetes Administrator (CKA) and Certified Kubernetes Application
Developer (CKAD), The Linux Foundation.
Education
MSCS, National University of Sciences and Technology (NUST), Islamabad, 2019 to 2021.
Thesis: GPU Cluster Optimization for ML Workloads.
BSCS, FAST-NU, 2015 to 2019.
OpenStack (Nova, Neutron, Cinder, Keystone, Glance, Horizon, Ceilometer) with Ceph
storage, OVS, VXLAN and DVR networking, VMware, Proxmox, and VPC and CIDR design.
Also worked in production
AWS including Bedrock, EKS, Redshift and Glue, Azure, GCP, IBM Cloud, Kamaji,
MetalLB, K3s, Rancher, Gateway API, Istio, Vault and KMS, GitHub Actions, GitLab CI,
Jenkins, Argo Workflows, PyTorch, Marvin, Modal.
Assurance and compliance
HIPAA, SOC 2, ISO 27001, air gapped and data sovereign delivery. Regression, load and
stress testing, plus penetration testing.
Community
I run internal AI, MLOps and agentic tooling workshops for engineering teams, and
write about operating AI systems in production.
08 Work with me
I am always open to collaboration, whether that is a senior, founding or forward deployed
engineering role, or a consulting engagement through Eprecisio.
I am most useful on the problems that appear after the prototype works. If your GPU bill
is growing faster than your usage, if several teams are fighting over one cluster, if
you need a platform delivered somewhere the cloud cannot reach, or if you want agents in
your operations without handing them production, that is the conversation I want.