Senior Generative AI Scientist II – Model Risk & Validation

This is a fully remote position, open to applicants in United States.

📋 Description

• Provide solutions that assist our clients in identifying payment integrity challenges, lowering healthcare process costs, or enhancing healthcare outcome quality.

• Perform independent validation of existing models for benchmarking, assessment, and effectiveness evaluation.

• Identify elements of model drift and associated data drift for effective model risk management (MRM).

• Utilize extensive expertise in AI/ML/GenAI model development, including practical experience in model construction and evaluation.

• Benchmark and, when necessary, reconstruct existing models using updated data and potentially more modern and effective algorithms.

• Proactively enhance model monitoring practices, including methods for model registration, management of model metadata, and conceptualizing related tools and techniques.

• Fulfill all responsibilities as delineated in the annual performance review and/or goal-setting processes.

• Undertake all special projects and additional tasks as assigned.

⛳️ Requirements

• A Graduate Degree in a quantitative field such as Computer Science/Engineering, Statistics, or Operations Research with a focus on Advanced Statistics, Machine Learning, and AI.

• Familiarity with cutting-edge natural language processing techniques, including transformers, fine-tuning LLMs, and deploying LLMs using tools like HuggingFace, Langchain, LLAMA/Mistral, and OpenAI, as well as vector databases.

• Over 5 years of practical data science/AI experience, utilizing standard machine learning and data science tools such as pandas, scikit-learn, keras, nltk, and TensorFlow/PyTorch, with GPU experience.

• A general understanding of Responsible AI (RAI), including explainability (XAI), AI NIST RMF, and associated AI risk management frameworks.

• Experience in evaluating models for bias and fairness, with a keen ability to identify bias in model design and data, as well as utilizing metrics like SHAP and LIME.

• Knowledge of suitable model metrics and techniques for managing, evaluating, and monitoring GenAI models and LLMs.

• Experience in creating production-grade machine learning deployments on AWS, Azure, or GCP.

• Proficiency with Apache Spark™ and handling large-scale distributed datasets.

• Ability to communicate technical concepts effectively to both technical and non-technical audiences is an advantage.

• A passion for collaboration, a learn-it-all attitude, and a commitment to driving value through AI.

🏝️ Benefits

• Coverage for medical, dental, vision, disability, and life insurance.

• 401(k) savings plans.

• Paid family leave.

• 9 paid holidays each year.

• 17-27 days of Paid Time Off (PTO) annually, based on specific level and tenure with Cotiviti.