Data Engineer 3

This is a fully remote position, open to applicants in California, +13 more states.

📋 Description

• Design, architect, and construct high-performance, scalable data pipelines for extensive data processing.

• Create complex Big Data ETL workflows utilizing Databricks Delta Live Tables.

• Develop and implement data lake architectures with Delta Lake and Apache Iceberg.

• Establish and uphold best practices in data engineering to ensure quality, integrity, reliability, and performance.

• Implement observability and monitoring measures, including data cataloging, lineage tracking, and metadata management.

• Engage in peer reviews, share knowledge, support production, resolve incidents, and manage changes.

• Apply best practices in code management and version control.

• Generate documentation such as data flows, lineage diagrams, and architectural specifications.

• Create unit and integration tests in adherence to TDD principles.

• Take ownership of critical batch ETL components and effectively communicate data models and design decisions to stakeholders.

• Balance the long-term architecture with immediate business requirements.

• Develop semantic layers and facilitate customer-facing self-service analytics solutions.

• Architect and design data pipelines to standardize, transform, and maintain data integrity.

• Engineer frameworks for data ingestion across various data types while monitoring data quality.

• Create data consumption methods including APIs, database views, and data extracts.

• Implement solutions across Kubernetes, Teradata, AWS, and Databricks platforms.

• Choose storage platforms based on data sensitivity, privacy requirements, and access needs.

• Apply data lineage and transformation rules for effective change management and issue resolution.

• Collaborate with cross-functional teams to enhance data sourcing, processing, quality, and efficiency.

• Exercise independent judgment and discretion in significant technical decisions.

• Maintain regular, consistent, and punctual attendance.

• Execute additional duties and responsibilities as assigned.

⛳️ Requirements

• Experience in Data Engineering.

• 5–7 years of relevant professional experience.

• Proficiency in Python and PySpark.

• Strong understanding of SQL.

• Hands-on experience in constructing large-scale data pipelines with Apache Spark.

• Experience in designing robust data architectures on AWS.

• Knowledge of data lake architectures and open table formats like Delta Lake and Apache Iceberg.

• Familiarity with Databricks Delta Live Tables.

• Preferable hands-on experience with Databricks and/or Snowflake.

• Understanding of data warehouse modeling paradigms.

• Comprehension of cloud-based architectures.

• Strong problem-solving abilities.

• Knowledge of semantic layer technologies such as Looker, Tableau, Snowflake, and Databricks.

• Experience with data quality, lineage, metadata management, observability, and monitoring.

• Familiarity with APIs, database views, and data extracts.

• Experience with Kubernetes, Teradata, AWS, and Databricks platforms.

• AWS Certified Developer Associate certification preferred.

• Databricks Certified Data Engineer Associate certification preferred.

• SnowPro Advanced: Data Engineer certification preferred.

• A Bachelor's degree is preferred; however, Comcast may consider coursework and experience or extensive related professional experience in lieu of a degree.

• Ability to maintain regular, consistent, and punctual attendance.

• Must be available to work nights, weekends, and varied schedules as needed.

• Capability to exercise independent judgment and discretion in significant matters.

🏝️ Benefits

• Bonus eligibility for most non-sales roles.

• Comprehensive Comcast benefits for eligible employees.

• Customized benefits options along with expert guidance.

• Continuous support tools for physical, financial, and emotional well-being.

• Opportunities to collaborate across various teams, locations, and resources.

• Flexible scheduling options for nights, weekends, or variable shifts as required.