Senior Data Engineer

This is a fully remote position, open to applicants in Florida, +4 more states.

📋 Description

• Construct data pipelines utilizing the medallion architecture.

• Create and enhance data pipelines with Databricks, assimilating data from diverse sources.

• Improve pipeline performance for increased efficiency and cost savings.

• Design and execute data models within the data lake setting.

• Establish data quality controls and validation methods throughout the data pipeline.

• Adhere to data governance policies and best practices.

• Integrate Databricks with Azure cloud services for storage, computing, and security, including Azure Data Lake Storage.

• Assist product development processes by ensuring accurate and dependable data flows.

⛳️ Requirements

• Bachelor’s degree in computer science, Information Technology, or a related discipline.

• Minimum of 7 years of experience in developing data pipelines using ELT/ETL.

• Proficient in working with Databricks, Spark, and Azure data services.

• Strong programming capabilities in Python, SQL, and PySpark.

• Experience in designing and maintaining configuration-driven pipelines that support Change Data Capture (CDC) and event/stream-based processing.

• Expertise in both NoSQL and SQL databases.

• In-depth understanding of data modeling, schema design, and optimizing database performance.

• Practical experience with a variety of file formats, including JSON, Parquet, and ORC.

• Hands-on experience in building and managing CI/CD pipelines (Azure DevOps, GitLab) and automating data workflow deployments.

• Strong grasp of data governance, lineage, and cloud security (Unity Catalog, encryption, and access control).

• Excellent analytical and problem-solving skills.

• Experience with data modeling tools and methodologies.

• Familiarity with cloud-based data solutions and platforms.

🏝️ Benefits

• Comprehensive full medical, dental, and vision insurance.

• Basic life insurance at no cost to the employee.

• Company-paid short-term and long-term disability.

• 12 weeks of 100% paid parental leave.

• Health Savings Account (HSA).

• Flexible Spending Accounts (FSA).

• Retirement savings plan.

• Personal paid time off.

• Paid holidays and a company-wide wellness day off.

• Paid time off to volunteer at nonprofit organizations.

• Pet-friendly office environment.

• Commuter benefits.

• Group pet insurance.

• On-the-job training and skills development.

• Employee Assistance Program (EAP).