Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Collaborate closely with the Data Engineering Lead to execute the target data architecture.

• Design and manage scalable data ingestion and transformation pipelines.

• Integrate data from various sources, including databases, APIs, SaaS applications, files, and cloud platforms.

• Implement data validation, quality checks, reconciliation, and monitoring processes.

• Conduct source data analysis and troubleshoot issues related to pipelines, data, and performance.

• Assist in proof of concepts (POCs), technical evaluations, testing, user acceptance testing (UAT), deployments, and stabilization of production environments.

• Engage in code reviews and maintain comprehensive technical documentation.

• Assume responsibility for assigned components and contribute to the overall solution delivery.

⛳️ Requirements

• 3–4 years of practical experience in Data Engineering, ETL/ELT processes, data integration, and the development of data pipelines.

• Familiarity with cloud-based data platforms and contemporary data architectures.

• Proficient SQL skills, including expertise in joins, CTEs, window functions, aggregations, transformations, and query optimization.

• Experience with Python for data processing, automation, pipeline creation, debugging, and error management.

• Knowledge of ETL/ELT processes, both batch and incremental processing, data modeling, data quality assurance, validation, and reconciliation techniques.

• Practical experience with AWS, Azure, or GCP, focusing on cloud storage, computing, and access management.

• Preference for experience with Snowflake; alternatively, experience with Databricks, Redshift, Synapse, or BigQuery is acceptable.

• Understanding of data loading, transformation, and performance enhancement techniques.

• Experience in integrating databases, APIs, SaaS applications, files, and cloud storage solutions.

• Knowledge of REST APIs, JSON, authentication mechanisms, and error handling practices.

• Familiarity with orchestration tools like Airflow, Azure Data Factory, AWS Glue, or similar platforms.

• Experience with Git, logging, monitoring, troubleshooting, and providing production support.

• Exposure to dbt, streaming/event-driven architectures, CI/CD practices, and Docker is preferred or advantageous.

• Understanding of analytical data modeling and datasets tailored for reporting and self-service analytics.

• Strong communication and problem-solving abilities.

• Capability to work independently while maintaining close collaboration with the Data Engineering Lead.

🏝️ Benefits

• Outstanding work environment with supportive colleagues.

• Open-door policy and ongoing connection with the leadership team.

• Opportunities for personal and professional growth through diverse learning initiatives, including code combat, meetups, and knowledge enhancement sessions.

• Collaboration with team members across various locations.

• Freedom to work autonomously and express creativity.

• Chance to make a significant impact.

• Opportunities for learning, upskilling, and leadership development.

• Commitment to equal opportunity and workplace diversity.