
Member of Technical Staff – AI Hardware, System Performance
Posted 1 day ago

Posted 1 day ago
This is a fully remote position, open to applicants in United States, +3 more locations.
• Construct foundational performance models for LLM inference and training.
• Examine arithmetic intensity, roofline performance, prefill and decode behavior, KV cache capacity, memory bandwidth, batching, parallelism, and latency-throughput trade-offs.
• Evaluate GPUs, TPUs, custom ASICs, emerging accelerators, HBM, on-chip SRAM, memory tiering, scale-up fabrics, scale-out networks, racks, pods, and datacenters.
• Analyze architecture and system design trade-offs related to compute, memory bandwidth, interconnect topology, power, cooling, and workload suitability.
• Link semiconductor manufacturing choices to hardware performance, availability, and cost.
• Investigate process nodes, die size, reticle limits, advanced packaging, 2.5D integration, 3D stacking, HBM integration, yield, and silicon economics.
• Review chip, system, rack, and cluster announcements utilizing specifications, die shots, rack layouts, benchmarks, and network diagrams.
• Develop independent assessments of real-world performance in comparison with vendor claims.
• Validate internal models against benchmarks and independently collected performance data.
• Explore discrepancies between theoretical peak performance and actual application performance.
• Convert technical analysis into metrics such as tokens-per-second-per-watt, per-dollar-of-capital-expenditure, and per-megawatt.
• Contribute to TCO, tokenomics, accelerator, memory, networking, and datacenter research.
• Publish comprehensive research on AI accelerators, systems, infrastructure, and performance.
• Serve as a technical authority during client engagements, briefings, and discussions with engineering and investment stakeholders.
• Collaborate with analysts focused on accelerators, memory, networking, semiconductors, and datacenters.
• Support SemiAnalysis products and research initiatives, including the Inference Simulator, InferenceX, Tokenomics Model, Accelerator & HBM Model, and AI Cloud TCO research.
• Potentially define and oversee a research coverage area based on technical expertise.
• Profound technical understanding of modern compute hardware at various levels of the stack, including silicon, systems, networking, clusters, or datacenters.
• Strong desire to learn unfamiliar aspects of the hardware and infrastructure stack.
• Excellent quantitative reasoning skills and the ability to apply first principles using FLOPs, bytes, bandwidth, latency, joules, watts, and dollars.
• Capability to construct structured models that link hardware design, workload performance, infrastructure requirements, and economic outcomes.
• Practical knowledge of AI workload requirements concerning memory bandwidth, memory capacity, interconnect communication, collective operations, and actual compute utilization.
• Working comprehension of semiconductor manufacturing and economics, including process-node trade-offs, die size, yield, die cost, packaging, and HBM integration.
• Ability to articulate architectural decisions and their impacts on manufacturing cost, performance, power, and scalability.
• Strong written communication skills tailored for engineering and investor audiences.
• Competence in navigating ambiguous technical inquiries from definition through analysis, validation, and publication with minimal supervision.
• Self-motivated, intellectually rigorous, detail-oriented, and comfortable in challenging assumptions.
• Genuine passion for computer hardware, semiconductor technology, AI systems, and infrastructure research.
• Hands-on experience with AI training or inference frameworks such as vLLM, SGLang, TensorRT-LLM, PyTorch, or JAX (preferred).
• Background in GPU programming or kernel development using CUDA, Triton, or HIP (preferred).
• Understanding of transformer inference mathematics, including per-token FLOPs, memory traffic, KV cache sizing, MoE routing, and quantization effects (preferred).
• Experience in designing, deploying, benchmarking, or operating large GPU or accelerator clusters (preferred).
• Knowledge of datacenter networking, rack-scale architecture, power distribution, cooling, or large AI infrastructure deployments (preferred).
• Familiarity with training performance metrics, parallelism scaling efficiency, communication overhead, and failure recovery (preferred).
• Experience with benchmarking heterogeneous hardware (preferred).
• Practical experience in semiconductor processes, foundry operations, product engineering, yield, packaging, or testing (preferred).
• Experience in die cost modeling, semiconductor manufacturing economics, advanced packaging, or teardown analysis (preferred).
• Ability to interpret die shots, packaging layouts, physical specifications, and teardown data (preferred).
• Prior technical publications, research papers, open-source contributions, benchmarks, hardware teardowns, or industry analyses (preferred).
• Experience in communicating technical findings to clients, executives, engineers, or institutional investors (preferred).
• SemiAnalysis LLC participates in E-Verify to confirm the employment eligibility of all newly hired employees.
• Flexible remote work arrangement.
• Option for in-office or remote work environment.
• Opportunity to collaborate with a global team of over 50 analysts.
• Opportunities to broaden expertise across the entire AI hardware stack.
• Opportunities to enhance technical writing and publishing abilities.
• Opportunities to establish oneself as a recognized technical authority.
• Opportunities to lead client briefings.
• Opportunities to define and manage a research coverage area.
• Chance to contribute to SemiAnalysis models, simulators, datasets, and research methodologies.
Greenhouse Software
Greenhouse Software
Mitsubishi Heavy Industries
Calligo
Get handpicked remote jobs straight to your inbox weekly.