
Senior Data Engineer
Posted 5 days ago

Posted 5 days ago
This is a fully remote position, open to applicants in Ukraine, +1 more country.
• Design, develop, and maintain scalable ETL/ELT pipelines capable of processing petabytes of data.
• Architect data solutions leveraging AWS, Databricks, and Snowflake technologies.
• Implement both real-time and batch processing workflows utilizing Apache Airflow.
• Optimize data storage and retrieval processes tailored for analytical workloads.
• Establish data quality frameworks and monitoring systems to ensure data integrity.
• Set up data governance standards and best practices for effective data management.
• Create automated testing and validation processes for data pipelines.
• Develop solutions for data lineage and metadata management.
• Lead technical design reviews and make architectural decisions.
• Mentor junior data engineers and promote best practices within the team.
• Drive the adoption of CI/CD practices for data pipelines.
• Utilize Cursor, Claude, and GitHub Copilot to enhance development speed.
• Implement AI-driven data quality checks and anomaly detection mechanisms.
• Stay updated with emerging AI tools and their applications in the field of data engineering.
• Work with extensive datasets to create scalable data infrastructure and pipelines for M Science, a data-driven research and analytics firm catering to financial institutions and corporations.
• 3-8 years of experience in data engineering.
• Proficient in SQL at an expert level, including complex queries, performance optimization, and window functions.
• Advanced knowledge of Python, focusing on object-oriented programming, async/await, and data processing libraries.
• Experience with Apache Airflow, including DAG development, custom operator creation, and workflow orchestration.
• Expertise in Databricks, specifically in Spark optimization, Delta Lake, and Unity Catalog governance.
• Familiarity with AWS services (S3, EMR, Lambda, Glue) or equivalent cloud platforms.
• Knowledge of data modeling techniques such as dimensional modeling, data vault, and modern data stack patterns.
• Experience in mentoring junior engineers and leading technical projects.
• Strong communication skills for effective cross-functional collaboration.
• Experience with Agile methodologies and project management practices.
• Ability to translate business requirements into technical solutions.
• 20 days of vacation.
• Public holidays observed.
• Mobilization reservation available upon successful completion of the probation period (for Ukrainian colleagues).
• Opportunity to exchange experiences and collaborate with talented colleagues.
• Competitive compensation for your contributions.
Highmark Health
Prompt Therapy Solutions Inc
Jones Lang LaSalle Americas, Inc.
Allshore Talent
Get handpicked remote jobs straight to your inbox weekly.