Senior Compute Infrastructure Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop the foundational systems layer for StackYak’s AI infrastructure product.

• Determine the balance between owning versus renting GPU capacity and establish standardization.

• Oversee Linux, kernel, firmware, drivers, CUDA, and/or ROCm management across various machines.

• Control decisions and delineate boundaries between bare metal, virtual machines, and containers.

• Architect and execute customer and workload isolation strategies.

• Streamline the automation of provisioning, rebuilding, recovery, and operation of machines.

• Integrate neoclouds and other computational service providers.

• Define hardware specifications and manage the process from purchase order to machine booting with vendors and data-center partners.

• Troubleshoot failures across hardware, firmware, operating systems, virtualization, storage, networks, and workload layers.

• Ensure the fleet is manageable by others, minimizing reliance on individual machine knowledge.

• Transform a mix of owned and rented GPU capacity into reliable, operational production infrastructure.

• Engage in technical decision-making and share responsibilities for production and on-call support.

• Transition seamlessly between system design, implementation, debugging, and operations.

⛳️ Requirements

• Experience running production Linux at a level where solutions were found in the kernel rather than documentation.

• Hands-on experience with GPU infrastructure for AI, HPC, cloud, or similarly intensive workloads.

• Successfully navigated NVIDIA and/or AMD platforms through challenging driver or firmware upgrades.

• Provisioned bare metal machines in significant quantities, where manual configuration was impractical.

• Managed machines that were not physically accessible and successfully recovered one in challenging circumstances.

• Made informed decisions between bare metal, KVM, and containers with tangible consequences, and articulated the reasoning.

• Developed Terraform and Python code that is critical for others in production environments.

• Enhanced host security and isolated workloads for customers with strict privacy requirements.

• Resolved failures that involved hardware, firmware, operating systems, drivers, networks, or workloads.

• Built or managed infrastructure within a neocloud, hyperscaler, GPU cloud, HPC environment, hosting company, data-center operator, or AI infrastructure company.

• Operated fleets that combine owned hardware with rented resources.

• Experienced with multi-GPU and multi-node systems.

• Knowledgeable in NUMA, PCIe topology, RDMA, NIC placement, and physical infrastructure layout.

• Specified or accepted hardware and comprehended the end-to-end process from purchase order to machine booting.

• Designed infrastructure capable of safely serving multiple customers.

• Experience working outside of narrowly defined enterprise teams.

• Ability to navigate across design, implementation, debugging, and operational roles.

• No specific degree, certification, or academic credential is necessary.

🏝️ Benefits

• Meaningful equity opportunities.

• Flexible remote and distributed work across various time zones.

• Direct access to the company's founders.

• Involvement in technical decision-making processes.

• Shared responsibilities for production and on-call duties.

• Compensation structure tailored based on location, engagement model, and experience level.

• Flexible employment or contract arrangements based on location.