Data Engineering Lead

This is a fully remote position, open to applicants in United States.

📋 Description

• Establish and uphold best practices and coding standards throughout the project.

• Perform comprehensive code reviews to guarantee compliance with established guidelines and maintain exceptional code quality.

• Provide clear instructions to team members while assisting in managing the daily operations.

• Maintain regular communication with the client.

• Design, develop, and support robust and scalable Spark applications.

• Produce clean, maintainable, and efficient code in accordance with best practices and coding standards.

• Enhance code for performance and scalability, ensuring effective data handling.

• Collaborate closely with cross-functional teams to deliver high-quality software solutions.

• Identify and troubleshoot technical issues, ensuring application reliability and performance.

• Develop and sustain comprehensive documentation for code, processes, and workflows.

• Provide technical leadership for the team by mentoring, coaching, and leading by example.

⛳️ Requirements

• Over 5 years of hands-on experience in software development.

• Extensive experience with Apache Spark, including platforms like Databricks and/or Azure Synapse/Fabric.

• Proficient in Python, with strong skills in data manipulation using tools like Pandas/Polars.

• Solid understanding of columnar data storage formats, especially Parquet, with practical experience using Delta Tables.

• Proven expertise in data processing, analysis, and transformation workflows.

• Strong analytical and problem-solving skills, with a keen attention to detail.

• A practical approach to balancing standardized processes with the flexibility needed to achieve project objectives effectively.

• Excellent organizational skills with the ability to self-manage, prioritize tasks, structure workload, and meet tight deadlines.

• Solid understanding of microservices architecture and its implementation in scalable systems.

• Nice to Have: Experience with Azure Cloud services (or other major cloud platforms), including various SaaS offerings such as Service Bus, Data Lake, Blob Storage, Redis, and more.

• Familiarity with FastAPI.

• Expertise in containerization and orchestration tools such as Docker and Kubernetes.

• Upper-Intermediate English proficiency.

🏝️ Benefits

• Opportunities to work on international projects.

• Options for in-office, hybrid, or remote work.

• Medical healthcare coverage.

• A recognition program to celebrate achievements.

• Opportunities for professional and personal development.

• Classes in foreign languages.

• A well-being program to support employee health.

• Corporate events for team building.

• Sports compensation to encourage active lifestyles.

• A referral program to reward employee recommendations.

• Provision of necessary equipment.

• Paid vacation and sick days.