Senior AI Engineer, MapGPT

This is a fully remote position, open to applicants in United States.

📋 Description

• Define the behavior of products, measurement frameworks, and evaluation systems for non-deterministic behaviors.

• Develop hypotheses, establish accurate results, compile datasets, and regulate changes based on regression outcomes.

• Regularly assess APIs, SDKs, data representations, and reference applications from the perspective of end users.

• Detect misuse, integration anti-patterns, and deficiencies, propose solutions, and ensure their implementation.

• Take ownership of the MVP in alignment with an agreed-upon north star technical design.

• Balance the pursuit of technical excellence with delivering valuable increments.

• Construct data pipelines and tools for data ingestion, conflation, entity resolution, quality checks, and both batch and streaming jobs.

• Monitor and assess external datasets, models, and benchmarks derived from research and open-source initiatives.

• Determine the appropriate time to utilize existing solutions versus developing internal ones.

• Design feedback loops and instrument systems to facilitate reproducible failure contexts.

• Transform recurring failures into evaluation cases.

• Establish boundaries for model-tools and enhance model harnesses.

• Define model responsibilities, delegation, and state management.

• Improve latency and cost per request through streaming, partial results, caching, model routing, and prompt structuring.

• Develop internal CLI, MCP, and other tools for rapid iteration.

• Share reusable tools with other teams.

• Conduct code and design reviews.

• Mentor engineers in evaluation practices.

• Assist in resolving unresolved technical issues.

• Participate in a 24/7 on-call rotation to ensure system availability.

⛳️ Requirements

• Bachelor's Degree in a STEM field.

• Over 5 years of experience in software engineering.

• Proven ownership of production services, pipelines, or SDKs.

• At least 2 years of experience delivering LLM-backed features to actual users.

• Familiarity with systems that manage error budgets, on-call rotations, and customer-facing regressions.

• In-depth data engineering skills, including SQL and at least one distributed processing framework.

• Experience with data pipelines where incorrect records are more critical than processing speed.

• Proficiency in tool calling and agent orchestration, addressing stale context, hallucinated arguments, silent partial successes, and unbounded loops.

• Strong expertise in Python or TypeScript.

• Ability to read code in various programming languages comfortably.

• Direct experience or a profound understanding of evaluations for non-deterministic systems.

• Experience constructing datasets and pinpointing failures through evaluations.

• Practical knowledge of more than one agent harness.

• Experience in diagnosing latency within a distributed request path.

• Capability to navigate ambiguity and make judgments to deliver narrow, functional solutions.

• Involvement in an on-call rotation, including possible immediate responses outside of regular working hours and on weekends.

• Experience with geospatial data, routing, geocoding, POI or address data, OpenStreetMap, or conflating overlapping sources (nice to have).

• Experience in designing public APIs or SDKs (nice to have).

• Familiarity with building against MCP or similar tool transports (nice to have).

• Experience in conducting evaluations in CI (nice to have).

• Background in automotive, in-vehicle infotainment, CarPlay, or Android Auto (nice to have).

• Experience in voice pipelines including streaming ASR, TTS, barge-in, endpointing, or wake word (nice to have).

• Experience working under limited compute, offline, or intermittent connectivity (nice to have).

• Experience in managing a product through its initial external integrations (nice to have).

🏝️ Benefits

• Supportive health care.

• Parental leave.

• Flexible work arrangements.

• Opportunities for teaching and learning.

• Private medical coverage for you and your dependents.

• Generous maternity and paternity leave policies.

• Inclusive fertility support.

• Contributions to a Lifestyle Spending Account for health, wellness, and personal growth.

• Mental health support for you and your dependents.

• Flexible paid time away.

• Company holidays.

• Generous absence policies.

• Dedicated paid time for volunteering.

• Competitive retirement plans with matching contributions.

• Hybrid/remote work options.