Principal Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the technical vision and architecture for scalable, resilient, and cost-effective data platforms and pipelines.

• Design comprehensive batch and streaming data systems, which encompass ingestion, transformation, modeling, storage, orchestration, serving, and consumption.

• Document architectural decisions and establish reusable patterns for Change Data Capture (CDC), event-driven flows, schema evolution, slowly changing dimensions, data quality, and multi-tenant or domain-oriented models.

• Lead technical design evaluations and identify as well as mitigate architectural risks and technical debt.

• Set engineering standards for development, testing, deployment, documentation, monitoring, incident response, lifecycle management, CI/CD, and infrastructure as code.

• Implement frameworks for data contracts, schema management, lineage, observability, freshness, completeness, accuracy, and reconciliation.

• Advocate for security, privacy, access control, governance, retention, and compliance throughout the data lifecycle.

• Define Service Level Indicators (SLIs), Service Level Objectives (SLOs), alerting mechanisms, runbooks, and post-incident learning practices.

• Act as a technical authority on cross-functional data architecture, tooling, platform, and investment decisions.

• Assess build-versus-buy options, technologies, and vendors; maintain architectural records, reference architectures, roadmaps, and technical standards.

• Lead the development of production-quality data pipelines, datasets, and platform capabilities.

• Collaborate with analysts, data scientists, software engineers, and product teams to deliver reliable data products.

• Propel modernization, legacy migrations, pipeline consolidation, and minimize manual operations.

• Mentor data engineers, oversee architecture reviews and workshops, and create reusable templates, implementations, and playbooks.

⛳️ Requirements

• Over 10 years of experience in software engineering, data engineering, platform engineering, or a related technical field.

• More than 5 years of experience in designing and operating large-scale data platforms, distributed data systems, or enterprise-level data products.

• Proficient in end-to-end system design, including defining requirements, evaluating tradeoffs, resilient architecture, and delivering to production.

• Extensive experience with lakehouse or warehouse platforms, batch and streaming processing, data modeling, orchestration, metadata management, and data governance.

• Strong expertise in SQL and at least one general-purpose programming language, such as Python, Java, or Scala.

• Familiarity with distributed data processing frameworks and technologies like Spark, Kafka, Flink, Snowflake, Databricks, BigQuery, or Redshift.

• Solid understanding of analytical, operational, and domain-oriented data modeling.

• Proven experience in establishing standards for testing, CI/CD, observability, infrastructure as code, reliability, and security.

• Capability to lead technical decisions across multiple teams without direct authority.

• Outstanding written and verbal communication skills.

• Experience in mentoring engineers and elevating technical standards throughout an organization.

🏝️ Benefits

• Comprehensive benefits package.

• Holistic programs aimed at mind, body, and lifestyle designed for overall well-being.

• Additional compensation may include bonuses, commissions, equity, and other benefits.