Lead Data Engineer

This is a fully remote position, open to applicants in Virginia.

📋 Description

• Design, develop, and execute batch and real-time machine learning pipelines that facilitate forecasting, marketing, revenue management, and customer behavior analytics.

• Create data solutions that enhance revenue optimization, personalization, and data-driven decision-making across various business functions.

• Apply software engineering principles throughout the processes of requirements gathering, design, development, testing, deployment, and monitoring.

• Develop and optimize scalable data ingestion, transformation, and processing workflows utilizing distributed computing frameworks.

• Utilize AWS Data Pipeline, AWS Glue, and AWS EMR to establish dependable, high-performance data infrastructure.

• Implement data modeling, feature engineering, and data quality validation for production-ready machine learning and AI solutions.

• Collaborate with data scientists, analysts, and business stakeholders to convert analytical requirements into deployable systems.

• Address production defects and execute long-term solutions.

• Guide and mentor junior engineers in big data engineering, AI-driven applications, version control, CI/CD, and DevOps automation.

• Ensure adherence to enterprise data governance, security, and privacy standards.

• Perform performance tuning of extensive pipelines.

• Investigate and suggest emerging technologies in AI, cloud, and data engineering.

⛳️ Requirements

• Master's degree or foreign equivalent in Business Analytics, Computer Science and Engineering, or a related field along with three years of experience in Data Engineering, Analytics, or a related occupation.

• Alternatively, a Bachelor's degree or foreign equivalent in Business Analytics, Computer Science and Engineering, or a related field with five years of experience in Data Engineering, Analytics, or a related occupation.

• Progressive experience in application development and object-oriented programming using Scala and Python.

• Experience handling large data sets, Hadoop, data lakes, and cloud technologies.

• Proficient in utilizing SQL and NoSQL databases, including Hive, AWS Redshift, Microsoft SQL Server, and MySQL.

• Familiarity with REST API integration.

• Experience employing AWS Glue, EMR, or Athena for scalable data engineering.

• Knowledge of implementing distributed computing frameworks like Apache Spark and Hadoop MapReduce.

• Experience using DevOps tools and CI/CD practices.

🏝️ Benefits

• Health insurance

• Dental insurance

• Vision insurance

• 401(k)

• Paid time off (PTO)

• Life insurance

• Disability insurance

• Equal Opportunity Employer, including disability/veterans