Data Engineering Lead

This is a fully remote position, open to applicants in United States.

📋 Description

• Establish and uphold best practices and coding standards throughout the project.

• Perform code reviews to ensure compliance with guidelines and maintain high-quality code.

• Provide clear instructions to team members and assist in managing daily operations.

• Engage in regular communication with the client.

• Design, develop, and sustain robust and scalable Spark applications.

• Produce clean, maintainable, and efficient code.

• Enhance code for optimal performance and scalability.

• Collaborate with cross-functional teams to deliver exceptional software solutions.

• Identify and troubleshoot technical issues to ensure application reliability and performance.

• Create and maintain thorough documentation for code, processes, and workflows.

• Offer technical leadership, mentoring, coaching, and lead by example.

⛳️ Requirements

• Over 5 years of practical experience in software development.

• Extensive background with Apache Spark, including Databricks and/or Azure Synapse/Fabric.

• Proficient in Python.

• Strong skills in data manipulation using Pandas/Polars or similar technologies.

• Knowledge of columnar data storage formats, especially Parquet.

• Practical experience working with Delta Tables.

• Expertise in data processing, analysis, and transformation workflows.

• Strong analytical and problem-solving capabilities.

• Detail-oriented approach.

• Ability to balance standardized processes with the flexibility needed to meet project objectives.

• Excellent organizational and self-management abilities.

• Capacity to prioritize tasks, organize workload, and meet tight deadlines.

• Understanding of microservices architecture and implementation in scalable systems.

• Upper-Intermediate level of English proficiency.

• Experience with Azure Cloud services or other major cloud platforms, including Service Bus, Data Lake, Blob Storage, Redis, and similar offerings.

• Familiarity with FastAPI.

• Proficient in Docker and Kubernetes.

🏝️ Benefits

• Remote work arrangement.

• GDPR-compliant handling and secure storage of candidate information.