Data Scientist, Databricks, Mosaic AI

This is a fully remote position, open to applicants in Virginia.

📋 Description

• Design, assess, and implement machine learning and generative AI models for a federal agency.

• Create and train machine learning models utilizing Azure Databricks.

• Monitor experiments, models, and versions using MLflow.

• Develop reusable features within the Databricks Feature Store.

• Serve models via Databricks Model Serving.

• Construct retrieval and agent applications employing Agent Bricks and Databricks AI Search with Azure OpenAI models through Microsoft Foundry.

• Assess model and agent performance utilizing MLflow evaluation, LLM judges, labeled test sets, and human review.

• Oversee production models for drift and quality assurance.

• Prepare documentation as requested during model reviews.

• Evaluate model quality pre- and post-release.

• Document each model's intent and constraints.

• Establish maintainable pipelines that teams can retrain and rerun.

• Collaborate with developers, engineers, data scientists, architects, and strategists throughout strategy and implementation tasks.

• Contribute to the development of the firm's capabilities, tools, and business lines.

⛳️ Requirements

• Minimum of 4 years of experience in data science or machine learning engineering, including deployment of models in production.

• Bachelor's degree.

• Proficiency in Python for data science and machine learning, including libraries such as pandas, scikit-learn, and PySpark.

• Practical experience with Azure Databricks, including MLflow for tracking experiments and managing models.

• Experience in deploying and monitoring models in production, covering aspects like model serving, versioning, and drift monitoring.

• Experience in building LLM applications, particularly retrieval-augmented generation (RAG).

• Ability to evaluate model and LLM outputs against established metrics, test sets, and human assessments.

• Knowledge of Structured Query Language (SQL) and Git-based version control.

• Experience delivering projects in a federal government context, adhering to its security, access, and review standards.

• Strong writing, PowerPoint, and Excel skills.

• U.S. citizenship required; dual citizenship and green card holders are excluded by contract terms.

• Must meet the necessary suitability, security, and access criteria.

• Preferred: Experience with a federal system accredited at Impact Level 5 (IL-5) or higher.

• Preferred: Familiarity with Agent Bricks, Databricks AI Search (formerly Vector Search), or MLflow 3 evaluation for generative AI.

• Preferred: Experience with an additional managed machine learning platform, such as Azure Machine Learning, Amazon SageMaker, or Google Vertex AI.

• Preferred: Knowledge of Azure OpenAI or other models within Microsoft Foundry.

• Preferred: Certification in Databricks or a Microsoft Azure AI or data science credential.

• Preferred: Experience in model risk management or model validation, including documentation as required for model reviews.

• Preferred: Understanding of AI-assisted developer tools.

🏝️ Benefits

• Comprehensive medical, dental, and vision coverage with the employee premium fully covered and half of dependent premiums.

• Employer-sponsored life insurance, accidental death coverage, and short-term and long-term disability insurance.

• 401(k) plan with employer matching up to 4% of salary.

• Unlimited paid time off.

• Sponsored professional certifications and ongoing education opportunities.

• Comprehensive onboarding process.

• Function rotation to enhance skills and broaden perspectives.