Lead Data Engineer – Platform

This is a fully remote position, open to applicants in Macedonia.

📋 Description

• Design, construct, and uphold scalable data engineering frameworks and platform utilities utilized by engineering teams.

• Create reusable patterns, templates, and abstractions to standardize and expedite delivery.

• Define and enhance platform architecture decisions, ensuring scalability, maintainability, and consistency.

• Design and execute CI/CD pipelines and automation frameworks to boost engineering velocity.

• Establish and enforce engineering standards for testing, code quality, deployment, and documentation.

• Identify and eliminate manual or repetitive tasks through automation and tooling enhancements.

• Incorporate AI-assisted development tools into engineering workflows to enhance productivity.

• Develop and sustain AI engineering assets such as coding guidelines, prompt frameworks, and reusable agent configurations.

• Oversee the development and operational support of core data transformation frameworks, including dbt Core at an enterprise scale.

• Investigate and resolve framework-level issues, including deployment failures, dependency conflicts, and production incidents.

• Support the onboarding and enablement of engineering teams adopting platform tooling.

• Serve as the primary technical contact for platform and framework-related inquiries.

• Collaborate with engineering teams to identify pain points and translate them into platform enhancements.

• Ensure that platform tooling adheres to security, compliance, and operational requirements.

• Conduct and facilitate code and design reviews across platform components.

• Monitor platform health, performance, and adoption, iterating based on feedback and metrics.

• Contribute to documentation, developer guides, and enablement materials to enhance usability and adoption.

⛳️ Requirements

• Strong proficiency in Python for developing frameworks, automation tools, and reusable components.

• Hands-on experience with Databricks, including notebooks, workflows, jobs, and Unity Catalog.

• Strong SQL skills and experience with distributed processing frameworks such as Apache Spark.

• In-depth experience with dbt Core, including project structure, models, tests, macros, and deployment at scale.

• Proven experience in designing and maintaining CI/CD pipelines, such as GitHub Actions, Azure DevOps, or GitLab CI.

• Experience with data engineering platform design, including scalable pipeline and workflow architectures.

• Strong understanding of software engineering principles (DRY, SOLID, modular design).

• Experience with version control systems and modern Git workflows.

• Familiarity with cloud platforms (preferably AWS) and infrastructure-as-code concepts (e.g., Terraform).

• Experience in implementing automated testing strategies (unit, integration, data quality).

• Strong understanding of platform monitoring, logging, and alerting practices.

• Experience in writing technical documentation and developer-facing guidance.

• Experience working in complex engineering environments with multiple teams.

🏝️ Benefits

• Private health insurance.

• Education program.

• Wellbeing program.

• Free beverages.

• Events.

• Competitive conditions.

• Challenging projects.

• Cool colleagues.

• Honest feedback.