Lead Data Engineer

This is a fully remote position, open to applicants in Poland.

📋 Description

• Lead a team responsible for the AI self-service portal and its associated microservices.

• Collaborate with a diverse group of engineers to enhance our data platform.

• Construct and maintain a distributed platform that supports all ExParte data.

• Engage primarily in our product, focusing on the data pipelines.

• Directly connect with internal teams.

• Assess software and implementation alternatives, documenting them for technical teams.

• Partner with data analysts to gather insights on potential data collection efficiencies and pinpoint automation opportunities for manual processes.

• Implement best qualitative practices in program design and development.

• Be part of a distributed team located in North America and Europe.

• Utilize Azure cloud and Databricks.

• Create technical architectures and specific implementations to address business needs.

• Direct the team’s software engineering best practices by documenting standards and conducting code reviews.

• Diagnose both new and existing code, providing feedback and solutions to structural issues within the codebase as they emerge.

• Consult on the feasibility of nonfunctional requirements and ensure the successful implementation of features while adhering to those requirements.

⛳️ Requirements

• 3+ years of experience with Spark, Python, or Scala.

• 3+ years of experience with RDBMS and T-SQL.

• 3+ years of experience with NoSQL and Data Lakes.

• Strong knowledge of map/reduce programming models.

• Proficient in writing production-quality code.

• Extensive expertise in database schema design, optimization, and scalability.

• Experience with Azure or AWS cloud-based service-oriented architecture.

• Solid understanding of the testing pyramid (unit, integration, black box, service).

• Experience working in an Agile/Scrum environment.

• Strong analytical and problem-solving abilities.

• Excellent time management and organizational skills.

• Capable of tackling challenging issues both independently and within a team setting.

• Quick learner with the ability to adapt to new technologies and environments.

• Strong communication skills.

• Nice-to-have's:

• Experience with Databricks or Azure ML.

• Experience applying machine learning algorithms to tackle complex data mining challenges.

• Familiarity with JavaScript/TypeScript, HTML5, and CSS3.

• Experience with .NET core and Entity Framework Core.

• Experience with Power BI Embedded.

• Understanding of cloud platforms and providers, particularly Microsoft Azure.

• Knowledge of CI/CD tools in Azure DevOps.

• Bachelor's Degree in Computer Science, Engineering, Mathematics, or a related field.

🏝️ Benefits

• Employees have the option to work remotely.