Member of Engineering – Evaluations, Engineering

This is a fully remote position, open to applicants in Europe.

📋 Description

• Create a Python framework that enables poolsiders to easily implement both internal and public benchmarks in a centralized manner.

• Develop and sustain a pipeline that executes distributed evaluations at scale.

• Work collaboratively with modeling and product teams to discover avenues for enhancing our experimentation and evaluation tools.

⛳️ Requirements

• Solid engineering background.

• Proven experience in leading software projects across various functions.

• Expertise in building highly reliable and thoroughly tested services.

• Familiarity with distributed systems.

• Experience with data pipelines and distributed processing.

• Knowledge of message queues and event-driven architectures, such as Kafka and Google Pub/Sub.

• Experience with cloud technologies, including managed services and storage on GCP, AWS, or Azure.

• Proficient in monitoring and alerting tools like Grafana, Prometheus, and Datadog.

• Preferred: Background in designing frameworks or tools for developers.

• A product-oriented mindset for developing software aimed at developers.

• Ability to collaborate effectively with researchers and ML engineers to uncover opportunities.

• Preferred: Experience in ML ops and data visualization platforms.

🏝️ Benefits

• Fully remote work with flexible hours.

• 37 days of vacation and holidays per year.

• Health insurance allowance for you and your dependents.

• Equipment provided by the company.

• Allowances for wellbeing, continuous learning, and home office setup.

• Regular team gatherings.

• A diverse and inclusive culture that prioritizes people.