AV Simulation Domain Expert – Senior Principal

This is a fully remote position, open to applicants in Illinois.

📋 Description

• Lead the technical vision for map-grounded world foundation models, focusing on how to condition generative video and world models using map data, driving data, and scenario semantics.

• Train, fine-tune, and adapt generative models (including diffusion, latent video, and transformer-based world models) for generating driving scenarios, emphasizing domain adaptation, controllability, and conditioning on structured inputs such as maps, trajectories, agent behaviors, weather, and lighting.

• Assess and enhance cutting-edge foundation models, including NVIDIA Cosmos / Cosmos-Transfer and similar open-source world models, to evaluate their suitability for AV training data generation.

• Oversee the entire ML lifecycle from end-to-end: data curation, model training, evaluation, iteration, and development of production-grade pipelines.

• Spearhead proof-of-concept projects that showcase map-grounded synthetic scenario generation in collaboration with key technology partners.

• Establish measurable success criteria that extend beyond visual realism, concentrating on the utility of ML training data, controllability, and sim-to-real transfer.

• Present POC results with definitive GO / PIVOT / NO-GO recommendations supported by quantitative evidence.

• Integrate generative world models with traditional simulation environments (such as CARLA, NVIDIA Drive Sim, AlpaSim) where structured, physics-based scenarios are necessary.

• Create and programmatically produce OpenSCENARIO / OpenDRIVE definitions that serve both conventional simulators and generative pipelines.

• Formulate the sim-to-real strategy: assess domain gaps, identify potential failure modes, and set acceptable thresholds for subsequent model training.

• Determine the criteria for 'good enough' synthetic data for AV perception and planning: identifying when photorealism is essential, when label consistency suffices, and when controllability is most critical.

• Develop validation frameworks that combine objective metrics (such as distribution coverage, label accuracy, FID-style measures, and downstream task performance) with expert evaluation protocols.

• Define sensor fidelity requirements, including noise models, lens distortion, and lidar return characteristics, along with guidelines on how generative models should or should not replicate them.

• Collaborate with ML research teams on aspects of generative model architecture, controllability, and conditioning strategies.

• Work alongside perception and planning teams to ensure that synthetic data significantly enhances real-world model performance.

• Convert business requirements into technical feasibility assessments for product and executive stakeholders.

⛳️ Requirements

• Over 5 years of combined experience in AV simulation, perception/ML for AVs, or robotics simulation, with significant exposure to both simulation platforms and ML model development.

• Practical experience with at least one major simulation platform: CARLA, NVIDIA Drive Sim, or an equivalent.

• Proficiency in OpenDRIVE and OpenSCENARIO: ability to author and generate scenario definitions programmatically while understanding map format specifications.

• Familiarity with AV testing workflows, including scenario-based validation, ASAM OpenX standards, and awareness of frameworks such as ISO 34502.

• Knowledge of scenarios that stress-test AV perception and planning systems, along with the reasoning behind them.

• Demonstrated experience in training deep learning models end-to-end, with clear ownership over data, training, evaluation, and iteration processes.

• Expertise in generative video, world models, or related generative AI research and engineering.

• In-depth working knowledge of diffusion models, latent video models, and/or transformer-based world models.

• Experience handling high-dimensional temporal or spatio-temporal data (such as video, multi-sensor fusion, or driving data).

• Strong foundational skills in Python and PyTorch; adept at building research-grade tools that can scale to production levels.

• Proven capability to transition ML models from research environments into production, while addressing real-world constraints, quality, and safety standards.

🏝️ Benefits

• Not specified