Member of Technical Staff – Inference Platform

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Take ownership of execution environments for models within the Platform team.

• Advance Sophos, the online and batch inference runtime developed on Kubernetes.

• Execute large batch inference on temporary jobs utilizing multi-dimensional admission control via Kueue.

• Develop and enhance the Sophos controller along with Kubernetes custom resources.

• Improve model inference engines and feature processing through vectorized, columnar operations.

• Efficiently serve graphs and data from Lance-based storage.

• Manage training, post-training, and fine-tuning jobs within Avra's cloud and customer dataplanes.

• Propel autoscaling, GPU serving, performance, and cost optimization using telemetry for each model.

• Address deterministic job sizing, checkpointing and recovery, customer encryption and isolation, input-file resilience, and automatic profiling.

• Attain 99.9% serving availability, minimal online-inference latency, high batch throughput, optimized costs, GPU utilization, and dependable job completion.

⛳️ Requirements

• Proven experience in running model serving or large-scale batch computing on Kubernetes.

• Experienced in building Kubernetes controllers or operators.

• Proficient in profiling and enhancing data-intensive Python pipelines.

• Strong understanding of cost and compute efficiency.

• Experience with production-quality code and participating in code reviews.

• Readiness to operate the systems you develop.

• Nice-to-have: Experience with Ray, Ray Serve, or KubeRay in a production setting.

• Nice-to-have: Familiarity with Kueue or other batch scheduling and admission-control systems.

• Nice-to-have: Knowledge in GPU serving and performance enhancement.

• Nice-to-have: Experience with Arrow, Parquet, Lance, or other columnar formats.

• Nice-to-have: Experience shipping software to customer-hosted Kubernetes environments.

• Nice-to-have: Familiarity with GCP/AWS, GKE/EKS, and experience in financial services or regulated settings.

🏝️ Benefits

• Comprehensive health and wellness programs.

• Opportunities for professional development and career advancement.

• Flexible work environment and remote work options.

• Competitive salary and performance-based bonuses.

• Collaborative and innovative company culture.