Linux Platform and Data Integrity Engineer
You build and run the Linux platform behind large data pipelines, and you make sure every record can be trusted. You keep provenance with each record, keep test sets sealed and apart, and check and sign every release and trace it back to its sources.
Duties
- Run data pipelines at scale on Linux servers, from capture through transformation to release.
- Keep provenance with every record, including its source, its capture time and each transformation applied to it.
- Build and run the API and the viewer through which people query records and see their sources.
- Keep sealed test sets on separate storage with separate access, so no training or tuning step can read them.
- Draw audit samples from each release, and track the error rates reviewers find.
- Compute checksums for every file and publish a signed manifest with each release.
- Automate builds, deployments and checks so each release can be reproduced from version control.
- Watch pipeline health and fix each failure at its cause.
What you bring
- Linux systems and data-platform experience.
- Python and shell scripting for automation.
- Experience with PostgreSQL or another relational database.
- Experience with checksums, signing and reproducible builds.
- Comfort with version control and automated deployment.
Nice to have
- Experience with Kubernetes and GitOps.
- Experience with object storage and large file sets.
- Experience with Sigstore, GPG or another signing tool.
- Experience building data viewers or internal web tools.
Pay and benefits
$126,000 to $176,000 a year
Full-time W-2 employment with health insurance, a retirement plan and paid leave.
No application deadline. We accept applications for this role on an ongoing basis.
Location
Remote, United States
Apply
We review applications as each team grows and contact you when there is a match.
Rillor is an equal opportunity employer. We consider qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status or any other status protected by law.
If you need an accommodation to apply, email careers@rillor.com.
Notices
Trademarks. NVIDIA, AMD, Ceph, Kubernetes and related product names are trademarks of their owners and are used only to identify products. No affiliation or endorsement is implied.