Data Engineer – Snowflake, Databricks, BigQuery

This is a fully remote position, open to applicants in India.

📋 Description

• Design and construct highly scalable data ingestion pipelines for structured, semi-structured, and unstructured data sourced from APIs, databases, and application logs.

• Create intricate ELT/ETL transformation models utilizing Python, SQL, dbt, Apache Spark, and PySpark.

• Architect and enhance enterprise cloud data platforms using Snowflake, Databricks, or BigQuery.

• Develop optimized schemas, clustering keys, partition strategies, and storage parameters.

• Implement automated data orchestration pipelines featuring workflow schedules, error-handling paths, and dependency graphs with tools like Apache Airflow, Prefect, or Mage.

• Set up data quality and validation checkpoints.

• Monitor data latency, validate structural constraints, check row balances, and enforce alerts for data anomalies.

• Enhance query performance and manage cluster costs.

• Audit resource usage, optimize inefficient SQL joins, manage micro-partitioning schemas, and address execution bottlenecks.

• Oversee data platform access and compliance measures.

• Configure row- and column-level security, data masking protocols, and RBAC access control policies.

⛳️ Requirements

• 4 to 8 years of experience in core database engineering or backend development.

• A minimum of 3 years dedicated to building and maintaining enterprise-scale data infrastructure.

• At least 3 years of focused data engineering experience in designing pipelines and analytics data warehouses.

• Required certification: Snowflake Certified Core Data Engineer, Databricks Certified Data Engineer Professional, or Google Cloud Certified Professional Data Engineer.

• Strong expertise in advanced SQL optimization.

• Proficiency in Python programming.

• Experience in relational/dimensional data modeling, including Star/Snowflake schemas and Data Vault.

• Familiarity with cloud storage configurations.

• Deep knowledge of distributed computing principles.

• Understanding of big data architectures.

• Awareness of data stream processing limitations.

• Insight into cloud resource pricing models.

• Preferred experience in managing large-scale migrations from legacy data warehouses to modern cloud data lakes.

• Preferred familiarity with streaming architectures using Apache Kafka, Flink, AWS Kinesis, Google Pub/Sub, or similar cloud-native event systems.

🏝️ Benefits

• Remote work arrangement.

• Contract employment.