Machine Learning Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Engage throughout the entire model lifecycle: data acquisition, training, validation, deployment, monitoring, and iterative improvement.

• Investigate and validate data both prior to and following modeling, and meticulously document all findings.

• Execute SageMaker training jobs and pipelines using Python.

• Monitor experiments using MLflow and validate models against reserved cohorts.

• Register, promote, and deploy model versions ensuring a clear lineage of data, code, and models.

• Manage real-time endpoints and batch scoring utilizing a feature store and Snowflake.

• Optimize latency, costs, and capacity.

• Oversee prediction quality, failure rates, and drift through the use of dashboards and alert systems.

• Conduct monthly production health reviews and blameless postmortems.

• Recommend subsequent model versions based on production evidence and contribute to design requests for comments (RFCs).

• Assist analysts, product, and data teams in comprehending and utilizing model outputs.

• Generalize models across various subjects and curricula without the need for code duplication.

• Re-score extensive student populations while live services remain operational.

• Run multiple model versions concurrently and safely deprecate older versions.

• Enhance training and serving efficiency while maintaining model functionality.

• Collaborate on a stream-aligned team alongside a senior or staff engineer.

⛳️ Requirements

• Proven track record of successfully delivering models or machine learning systems that others rely on.

• For IC4, generally a minimum of three years of practical experience or a doctorate coupled with relevant work experience.

• A Master’s or PhD in a quantitative discipline, including psychometrics, is preferred.

• Proficiency in idiomatic, typed, and tested Python.

• Experience with CI/CD processes, GitHub Actions, and infrastructure managed by Terraform.

• Strong SQL skills.

• A solid foundation in statistics, including knowledge of posterior, likelihood, and calibration plots.

• Skills in exploratory data analysis.

• Exceptional technical writing abilities.

• Familiarity with AWS, preferably SageMaker.

• Working knowledge of an experiment tracking tool such as MLflow.

• Capability to determine if work is ready for deployment and to substantiate this assessment with evidence.

• Desirable: experience in psychometrics or educational measurement, cognitive diagnostic models or Bayesian networks, probabilistic programming (e.g., Stan or PyMC), R, Snowflake, dbt, Terraform, GitHub Actions, knowledge graphs or ontologies, and K–12 assessment data.

🏝️ Benefits

• Salary range of $110,000 to $115,000.

• A comprehensive culture and benefits program (specific benefits available via the linked HMH page).