Lead Data Engineer

This is a fully remote position, open to applicants in United States, +1 more country.

📋 Description

• Provide guidance to clients, including data owners, analytical users, and executive stakeholders, on data architectures that are ready for AI.

• Convert business inquiries into analytical and data engineering solutions.

• Operate independently or lead small teams on intricate data engineering projects across various industries.

• Design and establish AI-optimized data platforms, such as cloud data warehouses, lakehouses, ETL/ELT pipelines, orchestration tasks, and analytical layers.

• Develop semantic and analytical layers for platforms like Snowflake Cortex, Databricks Genie, BI tools, and AI copilots.

• Utilize Snowflake, Databricks, dbt, Fivetran, and cloud-native orchestration frameworks to deliver scalable solutions.

• Create ETL/ELT pipelines for structured, semi-structured, and unstructured data.

• Design data models that emphasize metrics layers, knowledge graphs, and semantic consistency.

• Produce production-ready SQL, Python, and Spark code while applying Git and CI/CD best practices.

• Implement AI-assisted methods for data exploration, quality assessments, schema creation, documentation, lineage tracking, and transformation acceleration.

• Contribute to internal accelerators, design patterns, and client-ready architectures within the data engineering and infrastructure practice.

• Work in collaboration with analytics, data science, and ML teams to operationalize AI-enabled analytics, features, and inference pipelines.

• Assist in practice development, business development, and innovative initiatives within the company.

• Travel as necessary; development of machine-learning models or algorithms is not included in this role.

⛳️ Requirements

• A degree in Computer Science, Engineering, Mathematics, or equivalent professional experience.

• Proven experience in managing stakeholders and collaborating with clients.

• Excellent written and verbal communication skills are essential.

• Over 5 years of experience working with relational databases and query languages.

• More than 5 years of experience in building production data pipelines involving structured, semi-structured, and unstructured data.

• At least 5 years of data modeling experience, including star schema, entity-relationship, or data vault methodologies.

• Over 5 years of experience writing clean, maintainable, and robust code in Python, Scala, Java, or similar programming languages.

• More than 5 years of experience with dbt Core/Cloud is preferred.

• Familiarity with AI tools such as Codex, Claude, Copilot, Snowflake Cortex Code, and/or Databricks Genie Code to enable data platform engineering workflows is preferred.

• Capability to manage an individual workstream independently and lead a small team of 1–2 members.

• Strong expertise in software engineering principles and best practices.

• Required experience in DevOps.

• Experience with cloud data warehouses, specifically Databricks or Snowflake, is essential.

• One or more certifications in Databricks or Snowflake are strongly preferred.

• Familiarity with cloud ETL/ELT tools such as Fivetran, dbt, Matillion, Informatica, or Talend is preferred.

• Experience with cloud platforms such as AWS, Azure, or GCP and container technologies like Docker or Kubernetes is preferred.

• Experience with Apache Spark is preferred.

• Experience in preparing data for analytics and adhering to a data science workflow is preferred.

• Prior consulting experience is strongly preferred.

• A willingness to travel is required.

🏝️ Benefits

• Fully remote work arrangement.

• Opportunity to work at the new headquarters in Sandy Springs, GA, for applicants based in Atlanta.