This is a fully remote position, open to applicants in United States.

• Design and uphold scalable ETL/data pipelines utilizing Databricks and AWS Glue.

• Create data ingestion and transformation workflows for both structured and semi-structured data.

• Develop efficient PySpark, Python, and SQL code for processing large volumes of data.

• Consolidate data from various sources, including MongoDB and relational databases.

• Execute data quality, validation, and reconciliation checks.

• Enhance pipeline performance, reliability, and scalability.

• Work in collaboration with architects, analysts, and application teams to provide data solutions.

• Assist with production deployments, monitoring, and troubleshooting.

• Over 5 years of experience in Data Engineering or ETL development.

• Extensive hands-on experience with Databricks and PySpark.

• Familiarity with AWS Glue and AWS data services.

• Proficient in Python and SQL.

• Experience working with MongoDB.

• Understanding of ETL, data modeling, and data warehousing principles.

• Preferred knowledge of Delta Lake/Lakehouse architecture.

• Experience with Git and Agile development methodologies.

• Comprehensive health and wellness programs.

• Competitive salary and performance-based bonuses.

• Opportunities for professional growth and development.

• Flexible working hours and remote work options.

Data Engineer

People also viewed