Senior Data Engineer

This is a fully remote position, open to applicants in Bulgaria.

📋 Description

• Design, develop, and enhance modern cloud-based data platforms that support analytics, AI, and data products.

• Create reliable batch, streaming, and near-real-time data pipelines utilizing Spark and Delta Lake.

• Construct workflows for data ingestion, transformation, and curation for both structured and unstructured data.

• Implement lakehouse methodologies and medallion layering across bronze, silver, and gold levels.

• Participate in cloud-to-cloud migrations, including re-platforming of pipelines and data models.

• Provide production-quality datasets for analytics, machine learning, causal modeling, and optimization efforts.

• Develop data pipelines tailored for GenAI applications, such as LLMs, RAG, vector-based data flows, and agent-based systems.

• Utilize Databricks MLflow and Unity Catalog or similar tools.

• Design both logical and physical data models.

• Manage workflows using Databricks Workflows, Airflow, dbt, or equivalent platforms.

• Apply event-driven and streaming architecture patterns.

• Ensure data governance, lineage, quality, and access control through tools like Unity Catalog, AWS Lake Formation, Microsoft Purview, or GCP Dataplex.

• Establish observability for data freshness, pipeline dependability, and adherence to SLAs.

• Enable serving layers such as APIs, feature inputs, and analytical endpoints.

• Enhance pipelines and infrastructure for improved performance, scalability, and cost efficiency.

• Understand stakeholder requirements, challenge expectations, and propose superior alternatives.

• Collaborate with data scientists, machine learning engineers, analysts, platform teams, and business stakeholders.

• Support the adoption of data products and contribute to the development of best practices.

⛳️ Requirements

• Proven experience as a Senior Data Engineer.

• Strong hands-on expertise with Apache Spark and Delta Lake.

• Proficient in Python and SQL.

• Documented experience in developing batch and streaming pipelines along with production-grade data platforms.

• Solid comprehension of data modeling, data quality, and governance principles.

• Significant hands-on experience with the Databricks Lakehouse Platform.

• In-depth knowledge of at least one major cloud provider: AWS, GCP, or Azure/Fabric.

• Willingness and capability to work across multiple cloud environments.

• Familiarity with lakehouse architectures and distributed systems.

• Strong grasp of scalability, dependability, and performance metrics.

• Naturally inquisitive, with excellent problem-solving skills and a collaborative mindset in cross-functional teams.

• Comfortable challenging business requirements.

• Adaptable and enthusiastic about working across various platforms.

• Experience in agile methodologies or consulting is advantageous.

• Nice-to-have: experience with multiple cloud environments or cloud migration.

• Nice-to-have: familiarity with Snowflake.

• Nice-to-have: experience with GenAI and AI data systems, including RAG pipelines, vector databases, and LLM data preparation.

• Nice-to-have: knowledge of CI/CD for data pipelines and infrastructure-as-code, including Terraform and CloudFormation.

• Nice-to-have: expertise in streaming technologies such as Kafka, Spark optimization, or ML/experimentation platforms.

• Nice-to-have: experience in developing data products or large-scale analytics platforms.

🏝️ Benefits

• Flexibility with hybrid work options and 25 vacation days to ensure a healthy work-life balance.

• Co-subsidized transportation and Multisport cards.

• Premium health insurance for quick and easy access to top-tier healthcare services.

• Training policy covering technical and skills-related events, courses, and certifications.

• Personal career development roadmap shaped by performance evaluations.

• Self-care program offering psychological consultations and discussions for you and your team.

• Comfortable office space designed to enhance productivity.

• Engaging team events and company gatherings.

• Opportunities for growth.

• Values-driven organizational culture.

• International career opportunities.

• Inclusive culture with accessibility support.