Data Engineer – GCP, AWS, Databricks

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and maintain scalable ETL/ELT data pipelines.

• Create cloud-based data solutions utilizing GCP and AWS services.

• Leverage Databricks, Apache Spark, and PySpark for extensive data processing.

• Integrate data that is structured, semi-structured, and unstructured from various sources.

• Develop and enhance batch and real-time data-processing workflows.

• Optimize pipeline performance, reliability, scalability, and cost-effectiveness.

• Implement data-quality checks, monitoring, security, and governance standards.

• Design and support cloud data warehouses and data lakes.

• Troubleshoot production issues and conduct root-cause analysis.

• Collaborate with data architects, analysts, application teams, and business stakeholders.

• Create and maintain technical documentation for pipelines, data models, and workflows.

⛳️ Requirements

• A minimum of 6 years of experience is required in the position overview.

• At least 5 years of professional experience in data engineering.

• Strong practical experience with both GCP and AWS.

• Expertise in Databricks, Apache Spark, and PySpark.

• Proficient programming skills in Python and SQL.

• Demonstrated experience in developing ETL/ELT pipelines and cloud data platforms.

• Familiarity with data warehouses, data lakes, and dimensional data modeling.

• Experience with orchestration tools such as Apache Airflow or Google Cloud Composer.

• Knowledge of data security, governance, monitoring, and quality frameworks.

• Strong analytical, troubleshooting, and problem-solving abilities.

• Excellent communication and cross-functional collaboration skills.

• Preferred: experience with BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, Cloud Composer, S3, Glue, EMR, Redshift, Lambda, Kinesis, Delta Lake, Databricks Lakehouse, Apache Kafka, CI/CD, Git, Terraform, cloud infrastructure automation, and Agile development environments.

🏝️ Benefits

• Competitive salary and comprehensive benefits package.

• Opportunities for professional development and career advancement.

• Flexible work hours and supportive work environment.

• Access to the latest tools and technologies in data engineering.