Principal Data Engineer

This is a fully remote position, open to applicants in Poland.

📋 Description

• Take ownership of the comprehensive architecture for contemporary, cloud-native data platforms.

• Design scalable data ecosystems utilizing data mesh, data products, and data contracts.

• Make architectural choices spanning ingestion, storage, processing, and access layers.

• Ensure that platforms are secure, compliant, and production-ready.

• Design and implement cloud-native data platforms using Databricks, Snowflake, AWS, and GCP.

• Integrate with client systems to facilitate scalable, consumer-centric data access.

• Construct and optimize both batch and real-time data pipelines.

• Work with technologies like Kafka, Flink, Kinesis, and Pub/Sub.

• Orchestrate workflows utilizing Airflow, Dataflow, and Glue.

• Process and transform large datasets using Spark and Flink.

• Operate across relational, NoSQL, and analytical databases including Postgres, BigQuery, Snowflake, Cassandra, and MongoDB.

• Optimize storage formats and access patterns such as Parquet, Delta, ORC, and Avro.

• Implement secure and compliant data solutions with security integrated from the outset.

• Integrate data governance while ensuring developer velocity is maintained.

• Collaborate directly with clients to understand their challenges and shape effective solutions.

• Translate business requirements into actionable engineering decisions.

• Serve as a trusted technical advisor.

• Establish engineering standards, patterns, and best practices.

• Review designs and code, offering technical direction and mentorship.

• Enhance data quality, testing, observability, and operational excellence.

⛳️ Requirements

• Proficient in Python and SQL.

• Extensive experience with Spark and modern data platforms (Databricks / Snowflake).

• Solid understanding of cloud data services (AWS or GCP).

• Proven ownership of large-scale data platform architectures.

• Strong data modeling capabilities and architectural decision-making skills.

• Ability to navigate trade-offs between performance, cost, and complexity.

• Experience in building and maintaining large-scale data pipelines in production environments.

• Familiarity with various storage technologies and formats.

• Experience with Infrastructure-as-Code using Terraform or Pulumi.

• Proficiency in CI/CD pipeline processes utilizing tools such as GitHub Actions or ArgoCD.

• Experience with data testing and quality frameworks like dbt, Great Expectations, or Soda.

• Background in consulting or professional services settings.

• Ability to challenge assumptions and guide clients toward improved outcomes.

• Comfortable mentoring senior engineers and influencing the technical culture.

🏝️ Benefits

• Engaging, high-impact challenges.

• Opportunity to create modern platforms rather than maintaining outdated systems.

• Collaborate with senior engineers.

• Autonomy and influence in your role.

• Opportunities for personal and professional growth.