Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Act as a senior technical expert on client data initiatives.

• Assist in architectural design, oversee implementation, and ensure that delivered solutions align with commitments.

• Develop and implement scalable and reliable data pipelines and lakehouse architectures utilizing Databricks and cloud platforms.

• Write and audit production-quality code.

• Convert complex client requirements into technical designs, reference architectures, and data models.

• Support the technical scope and timelines, identify obstacles, and collaborate with project managers and client stakeholders.

• Mentor junior and mid-level data engineers through collaborative coding, code reviews, and constructive feedback.

• Contribute to technical blogs, reference architectures, and internal documentation.

• Establish and uphold DataOps standards, including automated testing, observability, version control, CI/CD, and thorough documentation.

⛳️ Requirements

• Over 6 years of practical experience in data engineering, specializing in designing and delivering production-grade data platforms.

• In-depth knowledge of Apache Spark, including runtime internals, performance tuning, and optimization techniques.

• Proficient in writing production-quality code in Python.

• Experience with Scala is a significant advantage.

• Proven experience in building and implementing solutions on Databricks, encompassing Delta Lake architectures, Unity Catalog governance, and Databricks Workflows.

• Databricks certification is highly desirable.

• Familiarity with at least two major cloud platforms (AWS, Azure, GCP), with in-depth expertise in at least one.

• Knowledge of cloud-native services, such as AWS Redshift/Glue/S3, Azure Synapse/Data Factory/ADLS, or Google BigQuery/Dataflow/GCS.

• Experience with CI/CD processes for data pipelines, automated testing, observability, and infrastructure-as-code.

• Background in ETL/ELT design, data warehousing, lakehouse architecture, and data modeling.

• Strong communication abilities with both technical and non-technical stakeholders.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Generous paid time off and flexible working arrangements.

• Opportunities for professional growth and development.

• Collaborative and innovative work environment.