Senior Data Engineer – Databricks

atCívicaRemoteES flagSpainFull-timeData EngineerSenior€562/year

Posted 6 days ago

This is a fully remote position, open to applicants in Spain.

📋 Description

• Design and implement Lakehouse architectures on Databricks utilizing a Medallion architecture (Bronze/Silver/Gold).

• Develop batch and streaming ingestion and transformation pipelines using PySpark, Spark SQL, and Delta Lake.

• Implement incremental ingestion with Auto Loader and Lakeflow Connect, including Change Data Capture (CDC) and Slowly Changing Dimensions (SCD).

• Construct declarative pipelines using Lakeflow Spark Declarative Pipelines and orchestrate them with Lakeflow Jobs.

• Implement data governance with Unity Catalog, including permissions, lineage, catalogs, schemas, Lakehouse Federation, and Delta Sharing.

• Apply data quality processes, expectations, quality monitoring, and pipelines.

• Optimize the performance and costs of clusters, jobs, serverless environments, and SQL Warehouses.

• Integrate data from databases, files, APIs, and events in cloud environments.

• Participate in migrations from legacy platforms to Databricks.

• Apply best practices for CI/CD and infrastructure as code using Databricks Asset Bundles, Git, and Terraform.

• Support pre-sales processes through reference architectures, estimations, technical proposals, Proof of Concepts (PoCs), and client demonstrations.

• Mentor junior and semi-senior profiles.

• Document architectures and developments.


⛳️ Requirements

• More than 5 years of experience in data engineering.

• At least 2-3 years of experience in projects involving Databricks.

• Proficiency in PySpark, Spark SQL, and Delta Lake.

• Advanced knowledge of SQL and Python.

• Experience with Unity Catalog, Lakeflow Jobs, and Lakeflow Spark Declarative Pipelines or DLT.

• Experience with at least one cloud platform; Azure preferred, with experience in ADLS Gen2, Entra ID, Key Vault, and Data Factory; AWS or GCP is a plus.

• Competence in dimensional data modeling, both normalized and denormalized, along with DWH, DL, and DLH architectures.

• Experience with Git and CI/CD pipelines.

• Autonomy, proactivity, client communication skills, and a consultancy orientation.

• Ability to present and defend technical proposals in pre-sales.

• A passion for mentoring and an ability to adapt to various projects.

• Databricks Data Engineer Associate / Professional certifications are desirable.

• Experience with Snowflake, migrations to Databricks, dbt, Terraform, MLflow, Databricks SQL/AI/BI, Mosaic AI, or Vector Search is a plus.


🏝️ Benefits

• Permanent contract and stability from day one.

• Innovative projects using modern technology.

• Continuous training and professional development.

• Flexible budget for courses, certifications, and English classes.

• Communities and collaborative learning opportunities.

• 23 business days of vacation per year, plus an additional day for every two years of service, up to a maximum of 30 days.

• Remote work compensation of €562.50/year.

• Flexible compensation for training, meals, transportation, and childcare.

• Provision of PC, chair, monitor, keyboard, mouse, and footrest.

• A library of books and games available in the Granada office.

• Access to WellHub offering discounted sports activities.

• Discounts via IBenefits.

• Financial support for shared hobbies with colleagues.

• Surprise breakfast on your birthday.

• Celebration and support for the birth of children.

• Recognition for sporting challenges.

• Sports groups and cultural events.

• Monthly Civipregunta and Civicerves events covered by the company.

• Christmas lunch.

• Cívica Day for employees and their families.

People also viewed

Highmark Health11 hours ago

Data Engineer

US flagUnited States OnlyFull-timeData Engineer$86.4k – $138.6k/year
ApplyView job
Prompt Therapy Solutions Inc11 hours ago

Clinical Technical Support, Data Migration Specialist – B2B SaaS

US flagUnited States OnlyFull-timeData Engineer$65k – $75k/year
ApplyView job
Jones Lang LaSalle Americas, Inc.11 hours ago

Principal Data Engineer

US flagIllinois OnlyFull-timeData Engineer$196k – $235k/year
ApplyView job
Allshore Talent12 hours ago

Data Architect – Adobe Analytics, Customer Journey Analytics

EuropeFreelanceData Engineer
ApplyView job
Airbnb12 hours ago

Senior Data Engineer, Tech Foundations

US flagUnited States OnlyFull-timeData Engineer$196k – $230k/year
ApplyView job
Alight Solutions12 hours ago

Senior Director, Data Engineering & Analytics

US flagUnited States OnlyFull-timeData Engineer$170k – $250k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers