Senior Data Engineer

This is a fully remote position, open to applicants in Florida, +4 more states.

📋 Description

• Design and create scalable ETL and ELT pipelines based on Spark technology.

• Develop enterprise-level data products utilizing Medallion Architecture.

• Construct Bronze, Silver, Gold, and Semantic data layers.

• Engineer reusable data assets for enterprise-wide business use.

• Enhance the performance of large-scale distributed data processing tasks.

• Develop native solutions on Databricks employing Spark, Delta Lake, Unity Catalog, Workflows, SQL Warehouses, and Delta Live Tables.

• Improve cluster performance and workload efficiency.

• Implement automated deployment pipelines alongside CI/CD methodologies.

• Create scalable Lakehouse solutions.

• Design dimensional models tailored for business purposes.

• Construct Data Vault 2.0 models for integration across the enterprise.

• Develop semantic models and curated data products.

• Assist in Master Data Management initiatives.

• Establish enterprise data quality validation processes.

• Create metadata-driven engineering solutions.

• Facilitate lineage, governance, and traceability of data.

• Ensure adherence to production-ready engineering standards.

• Collaborate with Architects, Product Owners, Business Analysts, and Data Scientists.

• Engage in architecture and code review sessions.

• Provide mentorship to junior engineers.

• Promote engineering best practices throughout the organization.

⛳️ Requirements

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related discipline.

• Over 7 years of experience in Data Engineering.

• More than 3 years of practical experience with Databricks.

• Proficient in Apache Spark, particularly with PySpark and SQL.

• Extensive understanding of Delta Lake, Unity Catalog, Workflows, and Lakehouse architecture.

• Advanced skills in SQL development.

• Experience in constructing enterprise-level ETL/ELT pipelines.

• Familiarity with Azure cloud services.

• Strong background in Git, CI/CD, and DevOps methodologies.

• Experience in performance tuning and optimization.

• Preferred experience in the Insurance or Financial Services sector.

• Data Vault 2.0 certification or relevant implementation experience is a plus.

• Preferred familiarity with dbt, Terraform, and Azure DevOps.

• Experience in building metadata-driven frameworks is advantageous.

• Preferred experience in Master Data Management.

• Background in developing enterprise semantic models is desirable.

• Familiarity with data quality frameworks and data governance is a plus.

• Experience in enabling AI and analytics through trusted enterprise data products is preferred.

• Employment offer is contingent upon successful background screening.

🏝️ Benefits

• Eligibility for discretionary annual bonuses.

• Paid Time Off (PTO).

• Medical insurance coverage.

• Dental insurance coverage.

• Vision insurance coverage.

• Retirement savings plans.

• Disability insurance options.

• Life insurance coverage.