Lead Data Engineer

This is a fully remote position, open to applicants in Washington.

📋 Description

• Developing cutting-edge data pipelines and models that form the foundation of the studio's decision-making framework.

• Engaging in large-scale lakehouse and warehouse analytics systems that handle data feeds in both real-time and batch processing.

⛳️ Requirements

• Minimum of 5 years’ experience with SQL is essential.

• At least 5 years’ experience in designing and executing scalable ETL processes, including data movement and quality assurance tools.

• A minimum of 3 years’ experience with contemporary Big Data Analytics utilizing Data Lake, Spark, and file formats such as Parquet.

• Over 2 years’ experience in constructing cloud-based data systems, with a strong preference for Azure.

• Preferred Qualifications:

• • Experience in building data pipelines using Azure Databricks/Fabric/Spark.

• • Familiarity with working with data in delta lake format and utilizing Azure Data Explorer/Kusto.

• • Applying AI/ML techniques to data engineering scenarios (including feature engineering, feature stores, model training/serving datasets, and model monitoring data pipelines).

• • Experience with preparing and managing datasets for modern AI applications (such as LLM/RAG, experimentation/A-B testing, and privacy-conscious data access).

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health and wellness benefits.

• Opportunities for professional development and career advancement.

• Flexible working hours and a supportive work environment.