Senior Data Scientist, Agentic AI Systems

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop autonomous AI systems tailored for workflows in rare disease research, encompassing conversation logic, information-gathering protocols, and confirmation steps.

• Utilize Pydantic for modeling outputs, ensuring that structured outputs and tool calls maintain type safety, validation, and traceability to their origins.

• Author, version, and conduct regression tests on prompts related to clinical and scientific reasoning tasks.

• Create evaluation frameworks for tasks that lack a definitive correct answer, employing golden sets, offline regression suites, and model-based grading systems.

• Maintain responsiveness of multi-step LLM workflows under increased load through asynchronous design, concurrency management, streaming, timeout management, and handling of failures.

• Record system actions, request identifiers, latency, errors, and the rationale behind automated decisions for subsequent review.

• Convert the needs of researchers, clinicians, and the patient community into data models and system functionalities.

• Collaborate on manuscripts, conference abstracts, and posters with NIH investigators, receiving authorship recognition.

• Take ownership of significant components from the design phase through to deployment.

• Engage directly with NIH program staff, clinical geneticists, and specialists in rare disease information.

• Assist in establishing engineering standards for AI applications within the team.

⛳️ Requirements

• A Bachelor’s degree in Data Science, Computer Science, Bioinformatics, Biomedical Informatics, or a related discipline; equivalent professional experience may be accepted in lieu of a degree.

• An advanced degree is preferred.

• A minimum of 5 years experience in building and operating production software or data systems.

• At least 2 years of experience in deploying LLM-powered applications, including agents, retrieval systems, or evaluation processes.

• Proficiency in structured output and tool or function calling with typed schemas and validation mechanisms.

• Experience in evaluating systems that do not have a single right answer, utilizing golden sets, offline regression suites, or model-based grading systems.

• Capability to manage a service comprehensively, from schema design to deployment and ongoing operation.

• Eligibility to obtain and maintain a Public Trust Security clearance.

• Proficient in Python, FastAPI, Pydantic, and pytest.

• Expertise in LLM application engineering, including provider APIs and gateways, prompt and context design, structured generation, tool utilization, and tracing.

• Knowledge of PostgreSQL, including embeddings or vector search in conjunction with relational data.

• Familiarity with asynchronous and concurrent Python programming, as well as streaming results to clients.

• Experience with containers and Kubernetes.

• Proficient in Git-based collaboration and CI/CD practices.

🏝️ Benefits

• 100% coverage for Medical, Dental, and Vision for employees.

• Paid Time Off and Paid Holidays.

• 401K matching up to 5%.

• Educational benefits to support career advancement.

• Employee referral bonuses.

• Flexible Spending Accounts: Healthcare (FSA), Parking Reimbursement Account (PRK), Dependent Care Assistant Program (DCAP), and Transportation Reimbursement Account (TRN).