Site Reliability Engineer, GPU Infrastructure
You keep the GPU platforms Rillor runs reliable and measured. You set service level objectives, build the monitoring stack and its alerts, track capacity and lead the review after each incident until its cause is fixed.
Duties
- Set SLOs for platform services and model endpoints, and track error budgets against them.
- Build and run the monitoring stack of metrics, logs, traces and dashboards for clusters, GPUs and models.
- Write alerts that point to a cause and a runbook, and remove alerts nobody acts on.
- Track capacity across GPUs, storage and network, and project when each runs short.
- Lead blameless incident reviews, record causes and follow each fix until it ships.
- Automate the repairs that repeat, such as draining and returning failed GPU nodes.
- Run failure drills, such as losing a node or a storage path, and fix what they reveal.
- Work with platform engineering on changes that make the platform easier to run.
What you bring
- 3+ years in site reliability, platform or production engineering.
- A metrics and alerting stack such as Prometheus and Grafana in production.
- Kubernetes and Linux troubleshooting under load.
- Experience setting SLOs and running incident reviews.
- Coding in Go or Python.
Nice to have
- GPU telemetry with NVIDIA DCGM.
- OpenTelemetry tracing.
- Monitoring for model serving: latency, throughput and queue depth.
- CKA.
Pay and benefits
$124,000 to $201,000 a year
Full-time W-2 employment with health insurance, a retirement plan and paid leave.
No application deadline. We accept applications for this role on an ongoing basis.
Location
Remote, United States
Apply
We review applications as each team grows and contact you when there is a match.
Rillor is an equal opportunity employer. We consider qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status or any other status protected by law.
If you need an accommodation to apply, email careers@rillor.com.
Notices
Trademarks. NVIDIA, AMD, Ceph, Kubernetes and related product names are trademarks of their owners and are used only to identify products. No affiliation or endorsement is implied.