Member of Technical Staff – AI Hardware, System Performance

Posted 1 day ago

This is a fully remote position, open to applicants in United States, +3 more locations.

📋 Description

• Construct foundational performance models for LLM inference and training.

• Examine arithmetic intensity, roofline performance, prefill and decode behavior, KV cache capacity, memory bandwidth, batching, parallelism, and latency-throughput trade-offs.

• Evaluate GPUs, TPUs, custom ASICs, emerging accelerators, HBM, on-chip SRAM, memory tiering, scale-up fabrics, scale-out networks, racks, pods, and datacenters.

• Analyze architecture and system design trade-offs related to compute, memory bandwidth, interconnect topology, power, cooling, and workload suitability.

• Link semiconductor manufacturing choices to hardware performance, availability, and cost.

• Investigate process nodes, die size, reticle limits, advanced packaging, 2.5D integration, 3D stacking, HBM integration, yield, and silicon economics.

• Review chip, system, rack, and cluster announcements utilizing specifications, die shots, rack layouts, benchmarks, and network diagrams.

• Develop independent assessments of real-world performance in comparison with vendor claims.

• Validate internal models against benchmarks and independently collected performance data.

• Explore discrepancies between theoretical peak performance and actual application performance.

• Convert technical analysis into metrics such as tokens-per-second-per-watt, per-dollar-of-capital-expenditure, and per-megawatt.

• Contribute to TCO, tokenomics, accelerator, memory, networking, and datacenter research.

• Publish comprehensive research on AI accelerators, systems, infrastructure, and performance.

• Serve as a technical authority during client engagements, briefings, and discussions with engineering and investment stakeholders.

• Collaborate with analysts focused on accelerators, memory, networking, semiconductors, and datacenters.

• Support SemiAnalysis products and research initiatives, including the Inference Simulator, InferenceX, Tokenomics Model, Accelerator & HBM Model, and AI Cloud TCO research.

• Potentially define and oversee a research coverage area based on technical expertise.


⛳️ Requirements

• Profound technical understanding of modern compute hardware at various levels of the stack, including silicon, systems, networking, clusters, or datacenters.

• Strong desire to learn unfamiliar aspects of the hardware and infrastructure stack.

• Excellent quantitative reasoning skills and the ability to apply first principles using FLOPs, bytes, bandwidth, latency, joules, watts, and dollars.

• Capability to construct structured models that link hardware design, workload performance, infrastructure requirements, and economic outcomes.

• Practical knowledge of AI workload requirements concerning memory bandwidth, memory capacity, interconnect communication, collective operations, and actual compute utilization.

• Working comprehension of semiconductor manufacturing and economics, including process-node trade-offs, die size, yield, die cost, packaging, and HBM integration.

• Ability to articulate architectural decisions and their impacts on manufacturing cost, performance, power, and scalability.

• Strong written communication skills tailored for engineering and investor audiences.

• Competence in navigating ambiguous technical inquiries from definition through analysis, validation, and publication with minimal supervision.

• Self-motivated, intellectually rigorous, detail-oriented, and comfortable in challenging assumptions.

• Genuine passion for computer hardware, semiconductor technology, AI systems, and infrastructure research.

• Hands-on experience with AI training or inference frameworks such as vLLM, SGLang, TensorRT-LLM, PyTorch, or JAX (preferred).

• Background in GPU programming or kernel development using CUDA, Triton, or HIP (preferred).

• Understanding of transformer inference mathematics, including per-token FLOPs, memory traffic, KV cache sizing, MoE routing, and quantization effects (preferred).

• Experience in designing, deploying, benchmarking, or operating large GPU or accelerator clusters (preferred).

• Knowledge of datacenter networking, rack-scale architecture, power distribution, cooling, or large AI infrastructure deployments (preferred).

• Familiarity with training performance metrics, parallelism scaling efficiency, communication overhead, and failure recovery (preferred).

• Experience with benchmarking heterogeneous hardware (preferred).

• Practical experience in semiconductor processes, foundry operations, product engineering, yield, packaging, or testing (preferred).

• Experience in die cost modeling, semiconductor manufacturing economics, advanced packaging, or teardown analysis (preferred).

• Ability to interpret die shots, packaging layouts, physical specifications, and teardown data (preferred).

• Prior technical publications, research papers, open-source contributions, benchmarks, hardware teardowns, or industry analyses (preferred).

• Experience in communicating technical findings to clients, executives, engineers, or institutional investors (preferred).

• SemiAnalysis LLC participates in E-Verify to confirm the employment eligibility of all newly hired employees.


🏝️ Benefits

• Flexible remote work arrangement.

• Option for in-office or remote work environment.

• Opportunity to collaborate with a global team of over 50 analysts.

• Opportunities to broaden expertise across the entire AI hardware stack.

• Opportunities to enhance technical writing and publishing abilities.

• Opportunities to establish oneself as a recognized technical authority.

• Opportunities to lead client briefings.

• Opportunities to define and manage a research coverage area.

• Chance to contribute to SemiAnalysis models, simulators, datasets, and research methodologies.

People also viewed

Greenhouse Software11 hours ago

Software Engineer

CA flagCanada OnlyFull-timeSoftware EngineerC$105.8k – C$132.3k/year
ApplyView job
Greenhouse Software11 hours ago

Software Engineer

CA flagCanada OnlyFull-timeSoftware EngineerC$105.8k – C$132.3k/year
ApplyView job
Mitsubishi Heavy Industries11 hours ago

Senior Developer, Data Integration, BI

IN flagIndia OnlyFull-timeSoftware Engineer
ApplyView job
Calligo12 hours ago

Software Engineer

US flagUnited States OnlyFull-timeSoftware Engineer$120k – $150k/year
ApplyView job
Avient Corporation12 hours ago

Senior ADTS – Engineering, Chemistry, Specialist

US flagUnited States OnlyFull-timeSoftware Engineer
ApplyView job
ICF12 hours ago

Appian Developer

US flagUnited States OnlyFull-timeSoftware Engineer$67.4k – $114.5k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers